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INTRODUCTION 


Writing  in  1969,  George  Thayer  expressed  his  concern 

over  the  age  of  arms  and  the  total  grip  it  has  on  the  world. 

We  live  in  an  age  of  weapons .  Never  before 
in  the  history  of  mankind  have  the  weapons 
of  war  been  so  dominant  a  concern  as  they 
have  been  since  1945.  Armaments  now  have 
enough  destructive  power  to  destroy  most 
life  on  earth.  Their  acquisition  or  presence, 
in  a  large  part,  determines  the  makeup  of 
governments,  the  course  of  foreign  policy, 
the  thrust  of  economic  effort,  the  social 
climate  in  which  man  lives.  No  significant 
act  of  contemporary  history  is  free  of  their 
influence.  Few  other  concerns  in  the  world 
demand  so  much  effort,  time,  and  money. 1 

Seven  years  later,  Thayer's  comments  still  apply.  The  transfer 

of  arms  is  a  major  activity  in  the  international  arena  and 

continues  to  grow  in  magnitude. 

Until  recently,  little  formal  effort  has  been  given  to 
the  study  of  this  important  problem.  As  a  result,  little 
useful  empirical  theory  has  been  developed  which  can  be  used 
to  explain  the  effects  of  arms  transfers  and  guide  decision 
makers  in  the  formulation  of  policy.  Since  1969,  however, 
progress  has  been  made.  What  started  with  simple  descrip¬ 
tions  of  arms  flows  throughout  the  world  has  now  evolved 
into  a  fairly  sophisticated  analytic  effort  aimed  at 
simplifying  the  complexities  of  the  phenomenon,  uncovering 
regularities,  and  developing  theories. 


^Thayer,  George,  The  Arms  Business.  The  International 
Trade  in  Armaments ,  p"!  18 ,  Simon  and  Schuster,  1969 . 
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Despite  this  impressive  growth,  theoretical  development 
has  been  sporadic  because  of  a  lack  of  progress  in  developing 
indicators  and  techniques  which  meaningfully  measure  arms 
flow.  Money,  numerical  accounting  of  weapons  by  type,  and 
military  utility  all  have  been  used  to  operationalize  arms 
transfers.  Most  of  these  approaches  are  flawed,  however. 

This  thesis  attempts  to  deal  with  the  problem  of 
operationalization  by  demonstrating  several  different 
measurement  techniques  which  have  potential  utility  in  arms 
transfer  research.  Of  necessity,  the  process  begins 
modestly.  In  Chapter  I,  the  reader  is  introduced  to  basic 
measurement  theory  and  familiarized  with  important  concepts 
and  terminology.  In  addition,  the  importance  of  measurement 
is  discussed  particularly  as  it  relates  to  militairy  and 
political  analysis. 

Chapter  II  acknowledges  the  major  attempts  to  operation¬ 
alize  arms  transfers  and  tries  to  clarify  the  strengths  and 
weaknesses  inherent  in  each.  Particular  emphasis  is  placed 
on  past  attempts  to  measure  the  qualitative  differences  in 
arms  using  factor  analysis.  Although  critically  appraised, 
it  is  maintained  that  the  factor  analytic  approach  is 
valuable  because  it  strives  to  develop  a  meaningful  way  to 
compare  and  evaluate  military  capability.  A  closing  arg\iment 
is  made  for  the  importance  of  capability  assessment  and  its 
value  to  the  military  and  political  decision  maker  is 
stressed. 
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Following  Chapter  II 's  strong  endorsement  of  the 
capability  approach  to  operationalizing  arms  transfers, 
Chapter  III  explores  the  complexities  of  capability  analysis 
using  fighter  aircraft  as  a  model.  Two  conceptual  views  of 
capability  are  presented:  one  based  on  weapons  performance 
characteristics;  and  the  other  based  on  a  multidimensional 
evaluation  of  the  weapon,  operating  environment,  and  operator 
skill.  While  no  attempt  is  made  to  argue  in  favor  of  one 
approach  over  the  other,  care  is  taken  to  accentuate  the 
strengths  and  weaknesses  of  each  in  relation  to  capability 
measurement. 

Taking  the  two  conceptual  definitions  developed  in 
Chapter  III,  Chapter  IV  presents  four  scaling  techniques 
and  applies  them  to  capability  assessment.  Because  a  variety 
of  disciplines  are  represented,  the  reader  is  provided  with 
the  rationale  for  using  each  particular  approach  and  the 
theoretical  premises  behind  each.  A  major  contention  made 
is  that  judgemental  scaling  techniques  merit  strong 
consideration  as  a  means  to  operationalizing  capability. 

Chapter  V  serves  to  review  the  major  conclusions  of  the 
research  and  suggests  problems  that  need  to  be  addressed  in 
future  studies. 
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I.  BASIC  THEORY  AND  METHODS  OF  SCALING  AND  MEASUREMENT 


This  section  will  provide  definitional  and  background 
information  that  will  serve  the  reader  throughout  this  thesis. 
It  begins  with  a  discussion  of  the  orthodox  theory  of  measure¬ 
ment.  The  concepts  of  nominal,  ordinal,  interval,  and  ratio 
scaling  are  then  investigated,  with  particular  emphasis 
placed  on  the  level  of  information  assumed  in  each,  their 
interrelationships,  and  their  limitations.  Finally,  the 
reasons  why  measurement  is  important  for  the  analysys  of 
military  problems  are  explored  in  some  detail. 

The  classical  view  of  measurement,  which  receives  its 
fullest  expression  in  the  works  of  N.  R.  Campbell,  is  more 
restricted  than  the  accepted  view  advanced  in  current 
references.  For  Campbell  and  other  classicists  direct  or 
"fundamental”  measurement  is  possible  only  when  the  axioms 
of  additivity  are  isomorphic  with  the  manipulations  performed 
upon  objects.^  To  illustrate,  measurement  can  be  associated 
with  length  because  the  addition  of  two  lengths  results  in 
a  third  length  whose  magnitude  equals  the  sum  of  the  first 
two  and  which  makes  sense  within  the  context  of  the 
operation. 


^Churchman,  C.W.  and  Ratoosh,  Philburn  (ed.) ,  Measurement 
Definitions  and  Theories,  p.  22,  John  Wiley  &  Sons,  1959. 
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Adhering  to  such  a  constraint  would  mean  that  the  only 
other  things  which  could  be  measured  (aside  from  length) 
would  be  time  and  mass.  To  allow  for  quantification  of 
other  important  phenomena,  classicists  recognized  an 
indirect  or  derived  form  of  measurement  in  which  magnitudes 
are  defined  through  laws  relating  fundamental  magnitudes. 
Density,  the  ratio  of  mass  to  volume  (length) ,  is  thus  an 
example  of  indirect  measurement.  Notice  that  the  addition 
axiom  does  not  hold  since  adding  two  substances  with  equal 
densities  does  not  produce  a  substance  with  twice  the 
density.  The  one  aspect  which  does  link  direct  and  indirect 
measurement,  though,  is  the  implicit  understanding  that 
measurement  makes  sense  only  when  the  niambers  have  a  direct 
physical  interpretation. 

The  classical  view  of  measurement  was  challenged  in  1932 
when  a  panel  of  distinguished  British  scientists  discussed 
the  feasibility  of  quantitatively  estimating  sensory  events. 
Conservatives  argued  against  the  proposal  because  to 
accommodate  it  a  new,  more  general  theory  of  measurement 
would  have  to  be  accepted.  "Why,”  complained  Campbell,  "do 
not  psychologists  accept  the  natural  and  obvious  conclusion 
that  subjective  measurements  of  loudness  in  n\imerical  terms 
(like  those  of  length...)  are  mutually  inconsistent  and 
cannot  be  the  basis  of  measurement?"^ 


^Ibid. ,  p.  22. 
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While  physicists  and  others  scoffed  at  the  notion  of 
measuring  the  subjective,  the  growth  of  psychology  demanded 
the  precision  only  quantification  could  provide.  In  response 
to  this  demand,  S.S.  Stevens  proposed  four  scales  of  measure¬ 
ment  in  1946  which  were  differentiated  by  the  niamber  and 
type  of  mathematical  transformations  that  left  each  scale 
invariant.  The  greater  the  number  of  transformations  that 
could  be  applied  to  a  scale  without  altering  its  structure, 
the  less  precise  the  scale.  The  four  scales,  nominal,  ordinal, 
interval,  and  ratio,  all  have  endured  the  test  of  time  and 

4 

foinn  the  basis  of  modem  measurement  theory. 

The  nominal  scale  is  the  "lowest"  of  the  four  scales  in 
the  sense  that  no  assumptions  are  made  about  the  values  being 
assigned  to  the  data.  Any  one-to-one  mathematical  transfor¬ 
mation  (which  includes  all  of  those  mentioned  in  this  chapter) 
can  be  applied  without  distorting  any  information.  Values 
serve  merely  as  labels  for  distinct  categories  and  cannot 
be  used  to  rank-order  data  points  or  measure  the  distance 
between  them.  Nominal  scale  values,  in  other  words,  are 
symbols  which  indicate  common  class  membership  only,  and 
hence,  cannot  be  added,  multiplied,  or  manipulated  in  any 
other  way.  It  follows,  therefore,  that  statistical  techniques 


4 

Other  scales  have  been  developed,  most  notably  the 
logarithmic  interval  scale  and  the  ordered  metric  scale. 
Because  of  their  infrequent  use  they  are  not  considered, 
however . 
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which  depend  on  the  distance  between  data  points  or  on  its 
order,  e.g.,  mean,  median,  standard  deviation,  should  not 
be  used  to  describe  relationships  between  nominal  data. 

An  ordinal  scale  and  ordinal-level  measurement  results 
when  the  data  can  be  ranked  according  to  some  criterion. 

Since  the  only  thing  that  must  be.  preserved  is  the  rank- 
order  of  the  data,  any  monotone  increasing  transformation 
can  be  used  to  create  different  scale  values  if  desired 
without  distorting  the  scale.  While  empirical  operations 
of  "greater  than"  and  "less  than"  can  be  accomplished,  nothing 
can  be  said  about  the  distance  between  data  points  or  cate¬ 
gories  .  Thus ,  ordinal  scale  values  order  the  data  and  indi¬ 
cate  relative  magnitude  along  a  continuim,  but  they  do  not 
exhibit  any  other  properties  of  the  real  number  system. 

If,  in  addition  to  ordering,  distances  between  cate¬ 
gories  can  be  defined  in  terms  of  fixed  and  equal  units, 
interval  measurement  is  possible.  Linear  transformations 
of  the  form  x'=a+bx,  b>0  (where  x’  is  the  transformed 
scale  value,  x  is  the  original  scale  value,  and  a  and  b  are 
real  numbers)  can  be  used  to  adjust  scale  values  when  appro¬ 
priate,  Since  distances  between  points  can  be  calculated 
with  interval  values,  some  of  the  strongest  statistical  tools, 
e.g,,  standard  deviation,  product-moment  correlation,  and 
factor  analysis,  can  be  used  to  describe  the  data  or  to 
advance  theory.  In  other  words,  interval  measurement  allows 
more  subtle  relationships  to  be  explored  than  is  possible 
with  ordinal  and  nominal  measurement. 
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Finally,  the  highest  level  of  measurement  is  embodied 
in  the  ratio  scale.  Being  the  highest,  its  structure  remains 
invariant  only  with  similarity  transformations  of  the  form 
x'  =  bx,  b  >  0  (where  x'  is  the  transformed  value,  x  is  the 
original  scale  value,  and  b  is  a  real  number) .  The  ratio 
scale  retains  all  of  the  properties  of  an  interval  scale 
with  the  additional  feature  of  a  natural,  fixed  zero  point. 

This  is  significant  because  it  allows  for  ratio  comparisons 
between  data  points .  In  sum  the  values  on  a  ratio  scale 
exhibit  all  the  properties  of  real  numbers  and  afford  the 
greatest  flexibility  for  describing  and  reporting  relation¬ 
ships  within  a  set  of  data. 

A  summary  of  the  properties  of  Steven's  four  scales  can 
be  found  in  Table  I. 

Three  consequences  of  Steven's  scales  are  worth  empha¬ 
sizing  at  this  point.  First  the  range  of  operations  possi¬ 
ble,  from  basic  classification  at  the  nominal  level,  to  the 
extensive  mathematical  manipulation  possible  with  ratio 
values,  requires  a  general  definition  of  measurement.  For 
the  purpose  of  this  study,  therefore,  measurement  will  be 
considered  as  the  assigning  of  numbers  to  objects  (data) 
according  to  a  set  of  rules.  The  rules  can  be  as  general 
or  restrictive  as  the  circumstances  and  goals  of  the  research 
demand,  but  whatever  the  case,  they  must  be  applied  consistently. 

Second,  the  level  of  measurement  required  depends  on  the 
research  questions  asked.  If  the  analyst  wants  to  know,  for 
example,  whether  arms  transfers  to  a  certain  area  increase 
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TABLE  I 


5 


Scale 


Characteristics 


Transformations 
Leaving  Scale 

Form  Unchanged  Examples 


Nominal  Classification  or  Any  1-1 

grouping.  Determination  transformation 
of  equality  is  possible 

Ordinal  Order  or  ranking  where  Any  monotone 
instances  are  assumed  increasing 

to  be  on  a  continuum.  transformation 

Determination  of 
equality  and  of 
greater  or  less  is 
possible 


Interval  Arbitrary  base  (origin) .  Any  linear 

Arbitrary  unit  which  transformation, 

is  constant  over  the  a  +  bx,  b  >  0 

scale.  Determination 
of  (1)  equality, 

(2)  greater  or  less, 
and  (3)  the  equality 
of  ratios  of  intervals 
or  differences 


Ratio  Fixed  or  natural  origin.  Any  similarity 

Arbitrary  unit,  constant  transformation 
over  the  scale.  Deter-  bx,  b  >  0 
mination  of  (1)  equality, 

(2)  greater  or  less, 

(3)  equality  of  ratios 
of  intervals ,  and 

(4)  equality  of  ratios 


Numbers  on 

football 

players 

Military 

Rank 


Temperature 
(F  or  C) 
Quality 
point 
rating 

Factor  Scores 


Length, 

Cost 


^Adapted  from  Stevens,  S.S.  Handbook  of  Experimental 
Psychology,  Wiley,  1951. 
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or  decrease  conflict,  ordinal  measurent  is  adequate.  If 
the  problem  is  trying  to  ascertain  the  degree  to  which  con¬ 
ventional  arms  transfers  influence  conflict,  ratio  measure¬ 
ment  is  needed.  It  is  generally  felt  that  the  higher  the 
level  of  measurement  the  better  off  the  researcher  will 
be.  Assuming  the  time  and  resources  are  available,  this  is 
a  sound  operating  principle. 

Third,  the  use  of  a  particular  mathematical  model  or 

statistical  technique  to  describe  the  data  is  governed  by 

the  level  of  measurement  employed.  The  more  powerful  analy 

tical  tools  are  reserved  for  ratio  and  interval  data.  For 

most  statisticians,  this  is  an  immutable  law.  H.M.  Blalock 

for  instance,  states  that 

...it  is  not  legitimate  to  make  use  of 
a  mathematical  system  involving  the 
operations  of  addition  or  subtraction 
when  this  is  not  warranted  by  the 
method  of  measurement.  Ideally,  one 
should  make  use  of  a  data  gathering 
technique  which  permits  the  lowest 
levels  of  measurement,  if  these  are 
all  the  data  will  yield,  rather  than 
using  techniques  which  force  a  scale 
on  the  data . ° 

Surprisingly,  this  viewpoint  is  not  accepted  by  all 
statisticians.  John  Tukey  argues  that  the  fact  that  data 
are  collected  on,  for  example,  an  ordinal  scale,  should  not 
in  itself  restrict  the  analyst  to  low-level  analysis.  In 
fact,  when  using  judgmental  scales  to  quantify  subjective 


^Blalock,  H.M. ,  Social  Statistics,  pp.  17-20,  McGraw- 
Hill,  1972. 
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data,  there  are  methodological  justifications  for  using  more 
powerful  analytic  techniques,  most  notably,  the  Thurstonian 
laws  of  comparative  and  catagorial  judgment.  This  has  im¬ 
portant  consequences  for  arms  transfer  study  since  some  of 
the  measurement  techniques  explore  in  Chapter  IV  of  this 
thesis  are  judgmental. 

Tukey's  philosophy  is  not  a  carte  blanche  to  mate  low- 
level  measurement  with  high-level  statistics,  but  rather  an 
exhortation  to  glean  all  the  information  possible  from  the 
data.  The  tendency  when  dealing  with  "soft"  variables  is 
to  be  overly  conservative.  If  the  data  is  not  interval, 
it  is  categorized  as  ordinal.  As  Tukey  notes,  however,  the 
typical  state  of  knowledge  short  of  metric  (interval/ratio) 
information  is  not  rank-order  but  actually  something  more."^ 
The  analyst  should  be  attuned  to  this. 

It  is  important  to  realize  that  under  certain  circum¬ 
stances  the  level  at  which  data  is  measured  can  be  changed 
to  suit  the  purposes  of  the  research.  It  is  fairly  obvious 
that  ratio  or  interval  data  can  be  scaled  at  an  ordinal  or 
nominal  level  because  the  data  possesses  more  than  the 
req\iired  amount  of  information  and  precision.  This  is  not 
generally  done,  though,  since  such  a  conversion  would  result 
in  a  loss  of  information.  It  may  also  introduce  error  since 


^Abelson,  R.P.,  and  Tukey,  J.W. ,  "Efficient  Conversion 
of  Non-Metric  Information  into  Metric  Information,"  in  The 
Quantitative  Analysis  of  Social  Problems,  E.R.  Tufte  (ed.), 
p.  407,  Addison-Wesley ,  1970. 
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''artificial''  boundaries  are  imposed  on  the  data.  Less 
obvious,  and  of  more  potential  use  to  the  military  analyst, 
are  those  techniques  which  "up-grade"  the  measurement  from 
nominal  to  ordinal  or  ordinal  to  interval  levels.  Changing 
nominal  data  to  ordinal  can  be  accomplished  by  using  the 
Spearman  rank  correlation  if  a  suitable  reference  variable 

Q 

can  be  found.  Ordinal  to  interval  conversion  is  accomplished 

9 

through  class  ranking  and  expected  order  statistics.  The 
exact  procedures  involved  in  scale  conversion  are  covered 
elsewhere.  At  this  point  it  is  sufficient  simply  to  know 
these  maneuvers  are  possible. 

Thus  far,  the  emphasis  has  been  on  measurement  theory 
and  on  elucidating  the  characteristics  of  the  four  most 
commonly  used  scales  in  social  research.  An  obvious  question 
to  ask  at  this  point  is  "Why  should  an  analyst  —  and,  in 
particular,  the  military  analyst  —  be  concerned  with  measure¬ 
ment  in  the  first  place?"  It  is  important  to  suggest  some 
reasons . 

One  very  compelling  reason  for  such  emphasis  is  the 
nature  of  the  world  in  which  the  military  now  operates.  The 
problems  are  complex  and  the  quantity  of  information  immense. 
Computers  are  not  just  fashionable,  therefore,  but  are 
cibsolutely  essential  in  the  analysis  of  problems.  Being  able 


g 

Anderberg,  Michael  R. ,  Cluster  Analysis  for  Applications, 
p.  53,  Academic  Press,  1973. 

^Ibid. ,  pp.  56,  59. 
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to  describe  politico-military  variables  in  a manner  that  is 
amenable  to  computer  processing  thus  becomes  a  significant 
venture.  Measurement  helps  accomplish  this. 

Gurr  notes  that  the  ultimate  goal  of  almost  all  empirical 
research  is  the  development  of  empirical  theory. This  is 
a  central  concern  of  those  investigating  the  effects  of  arms 
transfers  in  various  parts  of  the  world  as  well.  Quantitative 
research  (and  hence  measurement)  contributes  to  theoretical 
development  by  increasing  precision  since  the  ability  to 
precisely  define  phenomena  allows  the  researcher  to  test 
theoretical  statements  more  rigorously  and  with  more 
assurance.  Thus,  in  an  important  sense,  a  fundamental 
prerequisite  for  the  continued  development  of  arms  transfer 
theory  is  measurement. 

Another  reason  scaling  and  measurement  are  important  is 
that  they  provide  standardization.  For  a  science  to  grow, 
other  researchers  must  be  able  to  replicate  previous  research, 
and  to  do  this  requires  reliable  measuring  procedures.  (This 
problem  currently  plagues  arms  research,  especially  when 
monetary  data  is  used.)  In  a  more  practical  sense, 
standardization  is  important  in  the  military  because  of 
frequent  changes  in  personnel  at  all  levels  and  in  all 
organizations .  A  related  advantage  is  that  by  being 
standardized,  a  scale  increases  objectivity  since  the  biases 


^^Gurr,  Ted  R. ,  Politimetrics ,  p.  6,  Prentice-Hall,  1972. 
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of  each  analyst  are  controlled  and  minimized.  Even  the 
biases  of  the  individual  who  devised  the  scale  are,  by 
virtue  of  the  openness  and  availability  of  the  scale, 
subject  to  great  scrutiny  and  control. 

Lastly,  scaling  and  measurement  permits  the  conciseness 
of  mathematics  to  be  used  in  describing  empirical  relation¬ 
ships,  expressing  theories,  and  reporting  results.  Addi¬ 
tionally,  since  mathematics  is  a  universal  language, 
measvirement  leads  to  greater  potential  communication  and 
utility. 

Measurement,  and  the  quantification  it  leads  to,  will 
not  guarantee  success  or  improve  analysis  unless  it  is  done 
correctly  and  effectively.  With  this  in  mind,  the  next 
chapter  presents  the  most  common  measurement  techniques  used 
in  the  study  of  conventional  arms  transfers  to  Third  World 
countries . 
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II.  CURRENT  WAYS  OF  MEASURING  ARMS  TRANSFERS 


Despite  the  excellent  work  done  by  Amelia  Leiss  and 
Geoffrey  Kemp  as  members  of  the  Arms  Control  Project  at  MIT, 
the  Stockholm  International  Peace  Research  Institute  (SIPRI) , 
and  others,  the  empirical  study  of  conventional  arms 
transfers  is  still  undeveloped.  As  mentioned  previously, 
one  factor  which  has  deterred  growth  is  a  lack  of  progress 
in  developing  indicators  and  techniques  which  meaningfully 
measure  arms  flow.  Meaningful  measurement  of  conventional 
arms  is  an  ambiguous  notion  that  must  be  defined.  In  this 
study  it  will  connote  assigning  numbers  to  conventional  arms 
according  to  a  set  of  rules  such  that  the  quantitative  and 
qualitative  dimensions  of  arms  flows  are  represented.  Such 
measurement  should  be  done  at  the  highest  level  possible  to 
minimize  restrictions  in  using  the  data  and  maximize  the 
statistical  tools  available  to  the  analyst. 

In  addition  to  these  requirements,  a  good  measurement 
procedure  must  have  validity  and  generate  pertinent  informa¬ 
tion  which  is  free  of  systematic  and  random  error.  In 
contrast  to  the  multiple  criteria  that  must  be  satisfied  to 
prove  indicator  validity  (e.g.,  face  validity,  convergent 
and  criterion  tests,  etc.),  the  validity  of  a  measurement 
technique  depends  essentially  on  the  soundness  of  the 
theoretical  principles  which  underlie  it,  as  well  as  on  its 
ability  to  provide  accurate  information  at  the  required 
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level  of  precision.  All  meastirement  theory,  it  should  be 
remembered,  espouses  a  set  of  rules  and  some/all  of  the 
properties  of  the  real  number  system.  Meeting  the  requirement 
of  acceptable  theory,  therefore,  usually  poses  no  problem. 

What  can  be  a  problem  is  insuring  that  the  scale  and  level 
of  measurement  is  appropriate  for  the  research  questions 
asked.  For  example,  questions  requiring  information  about 
general  trends  in  arms  transfers  can  be  accommodated  with 
nominal  or  ordinal  scales  and  indicators.  However,  when 
more  precise  information  is  required,  or  when  the  analyst 
wishes  to  operationalize  arms  transfers  for  a  multiple 
regression  analysis,  interval-level  measurement  is  required. 

In  sum,  no  matter  how  sound  the  measurement  theory,  the 
validity  of  a  measurement  technique  cannot  be  assessed 
apart  from  the  specific  research  questions  it  serves. 

The  measxjrement  technique  should  also  be  reliable;  that 
is,  repeated  measurement  of  the  same  object/attribute  should 
yield  consistent  results.  While  it  is  always  true  that  the 
measTiring  instrument  will  induce  some  error,  this  can  be 
minimized  by  insuring  that  the  instructions  and  rules 
comprising  the  measurement  procedure  are  sufficiently 
detailed  and  explicit  and  are  observed  religiously.  This  is 
especially  important  when  using  judgemental  measurement 
techniques  where  discriminal  differences  among  judges  are 
commonplace.  Again,  the  crux  of  reliability  is  replicability 
and  consistency  over  a  series  of  trials.  Any  technique 
which  cannot  provide  such  duplication  should  be  discarded. 
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There  are  formal  statistical  techniques  which  can  help 
the  analyst  measure  the  reliability  of  a  scale.  For  ordinal 
scales  (or  ordinal  ranking  which  is  assumed  to  have  an 
underlying  interval  measure) ,  the  Spearman  rank  correlation 
coefficient  (Rho)  or  Kendall's  tau  can  be  used  to  determine 
the  correlation  of  scaled  results.  For  judgemental  scales, 
split-halves  reliability  procedures  are  available.  The 
analyst  should  make  use  of  these  simple  statistical  tests  as 
a  matter  of  course. 

With  these  criteria  in  mind,  three  techniques  presently 
used  to  measure  arms  transfers  will  be  analyzed  in  this 
chapter:  dollar-value  measurement,  numerical/ inventory 

measurement,  and  capability  measurement. 

Traditionally,  the  most  common  technique  used  to  measure 
conventional  arms  flow  is  what  can  be  described  as  the 
dollar-value  approach.  Quite  simply,  analysts  using  this 
technique  quantify  the  volume  and  direction  of  arms  flow 
in  terms  of  the  dollar-value  of  the  weapons  systems.  SIPRI 
and  the  Arms  Control  and  Disarmament  Agency  (ACDA) • are  two 
organizations  in  particular  which  rely  heavily  on  this 
approach. 

Admittedly,  dollar-value  measurement  does  provide 
expedient  and  often  useful  information,  and  can  generally 
be  accepted  as  a  good  first  estimate  of  arms  levels.  There 


^^For  other  examples,  see  Lambelet  (1971) ,  Milstein 
(1970  and  1972),  and  Safran  (1969). 
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is  another  positive  feature  in  this  approach,  namely,  that 

12 

It  provides  ratio  measurement  and  thus  affords  the 
researcher  the  opportunity  to  compare  systems  through  the 
ratio  of  their  costs,  and  take  advantage  of  all  of  the 
features  of  the  real  number  system. 

Dollar-value  measurement  of  arms  can  also  be  used 
effectively  as  an  indicator  of  commitment  or  alignment 
between  nations,  or  as  a  variable  in  a  multiple  regression 
model,  J.S.  Odell's  work  correlating  U.S.  military 
assistance  (measured  in  U.S.  dollars)  with  recipient  nations' 
economic  value  to  the  U.S,,  is  an  example  of  such  usage. 

However,  dollar-value  measurement  can  be  misleading 
for  several  reasons.  In  the  first  place,  fluctuations  in 
arms  expenditures  do  not  always  reflect  the  actual  magnitude 
of  the  arms  transferred.  Occasionally,  increasing  expendi¬ 
tures  correspond  to  decreasing  numbers  of  weapons  actually 
transferred.  The  chief  reasons  for  this  are  inflation  and 
fluctuating  exchange  rates.  In  some  reports  this  disparity 
between  expenditures  and  the  number  of  systems  transferred 
is  partially  alleviated  by  adjusting  yearly  figures  on  the 
basis  of  some  arbitrarily  selected  exchange  rate.  Unfortunately, 


^This,  of  course,  ignores  arguments  in  value  theory 
which  suggest  that  money  has  different  psychological  value 
depending  on  the  amount. 

^^Odell,  J.S.  "Correllate  of  U.S.  Military  Assistance 
and  Military  Intervention"  in  Testing  Theories  of  Economic 
Imperialism,  ed.  by  S.J.  Rosen  and  J.R.  Kurth,  pp.  143-161, 
D.C.  Heath,  1974. 
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fluctuations  usually  occ\ar  too  rapidly  to  be  compensated  for 
by  this  procedure  and  information  about  a  weapon's  true 
value  at  the  time  of  the  transaction  is  obscured.  This  is 
especially  troublesome  when  using  dollar-value  figures  for 

14 

side-by-side  comparisons  of  several  countries'  arms  trade. 

A  second  weakness  in  the  dollar-value  approach  rests 
in  the  uncertainty  with  which  foreign  —  most  notably  Communist 
weapons  are  priced.  ACDA  admits  that  their  valuation  of 
Communist  arms  exports  reflects  Soviet  foreign  trade  prices 
which  tend  to  underestimate  the  value  of  the  equipment  in 
terms  of  Western  production  costs.^^  Moreover,  there  is 
insufficient  information  available  from  which  to  perform 
any  systematic  price  adjustments.  As  Sivard  notes,  "Although 
statistical  work  on  such  parity  rates  is  underway,  under 
international  sponsorship,  the  availability  of  purchasing 
power  parities  for  a  large  selection  of  coiintries  is  some 
distance  in  the  future. Hence,  there  is  acknowledged 
doubt  regarding  cost  data  for  foreign  arms  transfers. 

Third,  there  is  no  standard  formula  that  can  be  used 
to  determine  the  value  of  military  equipment  transferred 


^^World  Military  Expenditures  and  Arms  Trade  1963-1973, 
Washington:  Arms  Control  and  Disarmament  A-ency,  p.  1,  1973. 

^^The  International  Transfer  of  Conventional  Arms, 
Washington:  Arms  Control  and  Disarmament  Agency,  p.  1,  1973. 

^^Sivard,  Ruth  L.,  World  Military  and  Social  Expenditures 
1974 ,  p.  30,  WMSE  Publications ,  1^74 . 
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from  excess  stocks  or  equipment  that  is  no  longer  of  use 
to  a  supplier  country.  In  a  report  to  Congress,  ACDA  valued 

all  transfers  from  excess  stocks  at  "approximately  one-third 
17 

of  acquisition  cost.”  Michael  Mihalka,  however,  cites 
instances  where  excess  military  equipment  was  transferred 

to  other  nations  at  less  than  one-tenth  to  one-hundredth 
18 

the  initial  cost.  Clearly,  in  such  instances  it  is  vir¬ 
tually  impossible  to  determine  the  real  cost  to  the  buyer  and 
value  to  the  seller. 

Fourth,  in  transactions  with  most  Third  World  nations 
it  is  extremely  difficult  to  determine  exactly  how  arms 

deals  are  financed.  Leiss,  for  instance,  notes  that  barter 

19 

is  sometimes  a  part  of  arms  trade.  Political,  economic, 
and  other  concessions  may  also  be  involved  which,  although 
not  reflected  in  published  prices,  would  have  a  direct  but 
indeterminable  relationship  to  the  value  of  the  weapons 
(e.g.  the  Soviet  naval  base  at  Mursa  Matruh  in  Egypt) . 
Additionally,  internal  factors  common  to  many  Third  World 
countries  such  as  corruption,  inefficiency,  and  incompetence 
would  distort  the  figures  even  more.  The  analyst  clearly 


^'^Ibid. ,  p.  20. 
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Mihalka,  Michael,  Understanding  Arms  Acc;mulation; 

The  Middle  East  as  an  Example,  p.  14,  University  of  Michigan 
(Mimeo) ,  1973. 

19 

Leiss,  Amelia,  et.  al..  Arms  Transfers  to  Less  Developed 
Countries,  C/70-1,  p.  31,  MIT,  1970. 
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must  be  cautious  when  drawing  inferences  from  foreign  arms 
expenditure  data. 

Finally/  money  is  an  unreliable  indicator  of  the  quali¬ 
tative  differences  in  arms.  While  it  is  generally  true 
that  the  cost  of  arms  is  directly  related  to  their  level  of 
sophistication/  and  in  this  sense  to  their  quality /^^  the 
factors  discussed  previously  distort  this  relationship. 

Even  if  it  could  be  assumed  that  the  more  costly  the  weapon 
the  more  sophisticated  it  is/  this  does  not  necessarily 
relate  to  the  military  value  of  the  system.  Mihalka  [1973] 
points  to  the  F-111  and  the  C-5A  as  examples  of  very  costly 
weapons  systems  whose  military  value  has  been  relatively 
low.  In  terms  of  arms  trade  to  lesser  developed  countries/ 
this  problem  is  further  exacerbated  by  uncertainties  in  the 
capacities  of  the  countries  to  absorb/  maintain/  and  effec¬ 
tively  use  sophisticated  weapons  and  by  a  general  lack  of 
knowledge  about  spare  parts  and  required  training. 

To  summarize/  the  problem  with  the  dollar-value  approach 
is  not  the  level  of  measurement,  but  the  instability  of  the 
measurement  unit.  Reliability  is  a  serious  problem.  In  many 
instances  cost  does  not  even  depict  the  volume  of  arms 
transferred  accurately.  What  is  more,  it  fails  to  measure 
qualitative  differences  in  weaponry  and,  as  a  result,  cannot 
be  used  to  measure  military  balance  or  potential.  Cost  can 


good  example  of  this  is  the  Escort  Ship  Cost  Model 
(ESCOMO)  developed  by  R.  Wilson  at  the  Center  for  Naval 
Analyses . 
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be  used  as  a  variable  in  a  regression  model  or  as  an  indi¬ 
cator  to  explain  other  phenomena,  but  in  view  of  the  serious 
reliability  problems,  and  the  lack  of  standardization  in 
its  usage,  it  would  only  have  limited  utility. 

Most  analysts  relying  on  dollar-value  measurement  are 
well  aware  of  its  fallacies  and  usually  catalog  their  reser¬ 
vations  with  their  analyses.  Some  researchers,  such  as 
Amelia  Leiss,  have  taken  a  more  positive  approach  to  the 
problem  by  using  an  alternative  unit  of  measurement,  the 
weapon  system  itself.  In  terms  of  usage,  this  type  of 
measurement  —  referred  to  here  as  the  numerical/inventory 

21 

approach  —  is  almost  as  popular  as  the  dollar-value  approach. 
Insofar  as  the  numerical  inventory  method  reflects  actual 
weapons  amounts  transferred,  the  technique  unquestionably 
improves  measurement  reliability.  For  example,  it  is  able  to 
provide  a  more  accurate  accounting  of  the  volume  and  direction 
of  arms  traffic,  since,  by  focusing  on  the  actual  weapons 
themselves,  many  of  the  factors  which  distort  dollar  measure¬ 
ment  (such  as  arbitrary  exchange  rates  and  inflation)  are 
eliminated. 

Numerical/inventory  measurement  cannot  be  used  effectively 
to  describe  qualitative  differences  in  arms  transfers  because 
the  unit  of  measure  (the  weapon  itself)  does  not  embody  any 


^^See,  for  example,  Leiss  [1970],  Kemp  [1970],  and 
SIPRI  [1968-1974]. 
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general  attribute  or  characteristic  upon  which  to  base  com¬ 
parisons  with  other  weapons.  The  most  successful  measure¬ 
ment  of  qualitative  difference  in  arms  in  Leiss'  work,  for 
example,  occurs  only  when  she  constructs  indices  based  on 
some  inherent  weapons  characteristic.  The  best  example  of 

this  is  her  "modernity  index"  which  measures  the  "modernity" 

22 

of  aircraft  on  an  interval  scale.  It  appears  that  if  the 
qualitative  aspects  of  arms  must  be  measured,  some  abstract 
attribute  or  characteristic  must  be  identified  and  agreed 
upon  as  the  basis  for  comparison. 

The  fact  that  the  numerical/inventory  approach  cannot 
in  itself  represent  qualitative  differences  in  arms  makes  it 
inappropriate  as  a  measure  of  military  balance  or  capability 
an  issue  of  vital  importance  to  military  analysts  and  deci¬ 
sionmakers.  As  the  1967  Arab/Israeli  war  and  the  Vietnam 
rout  in  1975  both  show,  numerical  superiority  in  armament 
does  not  equate  directly  to  military  strength.  This  is  per¬ 
haps  an  obvious  point,  and  yet  it  is  frequently  forgotten 
or  distorted  in  arms  studies  because  most  comparisons  of 
the  military  strengths  of  coiintries  are  in  fact  done  on  the 
basis  of  numbers  of  systems.  The  inevitable  implication  is 
that  the  more  weaponry  a  country  has,  the  better  off  it  is. 
(Consider  The  Military  Balance  Series  published  by  IISS 
which  purports  to  be  "a  quantitative  evaluation  of  the 
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Leiss,  Amelia,  Changing  Patterns  of  Arms  Transfers ; 
Implications  for  Arms  Transfer  Policies,  C/70-2,  pp.  19-35 , 
219,  MIT,  1970. 
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military  power  . . .  throughout  the  world"  and  is  based  solely 
on  a  tabulation  of  men  and  equipment.)  A  definite  need 
exists  for  measurement  schemes  which  can  cope  with  the 
qualitative  factors  of  arms  analysis . 

A  promising  and  sophisticated  attempt  to  measure  the 
quality  of  conventional  arms  is  presented  by  Michael  Mihalka. 
In  search  of  a  better  indicator  than  military  expenditures 
and  nxamerical  inventories  to  measure  arms  accumulation, 
Mihalka  proposed  to  measure  weapon  system  capability.  The 
main  assumption  behind  such  measurement  is  that  any  weapon 
can  be  viewed  as  a  linear  combination  of  its  component  capa¬ 
bilities,  each  of  which  can  be  measured  at  an  interval  level. 
Thus,  by  selecting  the  appropriate  performance  characteris¬ 
tics,  it  is  possible  to  derive  a  numerical  value  which 
reflects  total  system  capability.  A  related  assumption  is 
that  all  the  characteristics  of  a  system  will  reduce  to  two 
underlying  dimensions  or  factors,  offensive  and  defensive. 
Intuitively,  this  is  quite  pleasing  since  military  planning 
and  tactics  are  dichotomized  the  same  way.  In  view  of  the 
salience  of  capability  as  an  attribute  of  arms,  and  the 
prospects  of  interval  measurement,  it  is  profitable  to 
examine  Mihalka 's  methodology  in  some  detail. 

To  begin  with,  selecting  appropriate  performance  varia¬ 
bles  is  based  on  three  considerations;  (1)  the  nxmber  and 
type  of  variables  deemed  necessary  to  define  the  weapon 
system  adequately;  (2)  whether  or  not  the  variable  is 
amenable  to  quantification;  and  (3)  the  availability  of  data. 
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The  second  consideration  is  particularly  striking  because 
it  implies  that  capability  is  an  inherent  attribute  of  a 
weapon  and  is  satisfactorily  described  by  "hard"  quantifi¬ 
able,  performance  data.  (Commitment  to  this  viewpoint  has  a 
profound  effect  on  the  acceptability  of  several  of  the 
measurement  techniques  proposed  later  in  this  study.) 

The  analytical  model  used  by  Mihalka  to  measure  the 
variables  is  the  oblique  multi-dimensional  factor  model 

which  identifies  main  factors  by  grouping  mathematically 

23 

related  input  variables  into  distinct  clusters.  Mihalka 's 
analysis  of  selected  variables  for  aircraft  results,  for  example, 
results  in  two  groupings;  (1)  speed,  technological  data, 
and  performance  —  defining  the  defensive  factor;  and 
(2)  payload  and  combat  radius  —  comprising  the  offensive 
factor.  The  following  data,  taken  from  Mihalka 's  study  of 
arms  accumulation  in  the  Middle  East,  describes  these  cluster¬ 
ings  in  terms  of  raw  factor  loadings.  Since  the  loadings 
are  the  correlation  coefficients  between  variables  and 
factors,  the  higher  the  value,  the  greater  the  correlation 
between  factor  and  variable. 


23 

Rummel,  R.J.,  Applied  Factor  Analysis,  p.  409, 
Northwestern  University,  1970. 
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TABLE  1 


Variable 


Factor  I  (Defensive)  Factor  II  (Offensive) 


Speed 

0.917 

Tech  date 

0.723 

Performance 

0.848 

Payload 

0.161 

Radius 

-  0.258 

0.015 
0.026 
-  0.051 
0.918 
0.719 


Since  each  factor  defines  a  group  of  interrelated 

characteristics,  it  can  be  considered  a  functional  unity 
25 

and  used  as  a  scale.  Factor  scores  are  derived  in  the 
following  way.  Each  variable  is  weighted  according  to  the 
statistical  variation  it  has  in  common  with  the  offensive 
and  defensive  factors  and  multiplied  by  the  data  value  of 
each  case.  The  sum  of  these  weight-times-data  products  for 
the  three  variables  comprising  the  defensive  dimension  yields 
the  defensive  factor  score.  Similarly,  the  sum  of  the 
weight-times-data  products  for  payload  and  radius  leads  to 
the  offensive  factor  score.  A  sample  of  the  aircraft  scores 
obtained  by  Mihalka  is  fo\ind  in  Table  II . 


^"^Mihalka,  op.  cit.  ,  p.  21. 
25Rummel,  op.  cit. ,  p.  30. 
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TABLE  II 


26 

DERIVED  CAPABILITY  SCORES 
FOR  AIRCRAFT 


Capability  Scores 


Name  of  System 

Manufacturer 

Defensive 

Offensive 

1.  Interceptors 

F-104A  Starfighter 

US 

1.410 

-  0.280 

Lightning 

UK 

2.554 

-  0.767 

MiG- 21  D/F 

USSR 

1.645 

-  0.795 

2.  Multipurpose 

Fighters 

F-4  C/D 

US 

1.695 

1.106 

F-5A 

US 

0.885 

0.040 

MiG- 19 

USSR 

0.997 

0.571 

Mirage  V 

France 

1.561 

0.311 

Mystere  IV  A 

France 

0.873 

-  0.593 

3.  Strike  Fighters 

A-4F 

US 

0.631 

0.787 

SU-7B 

USSR 

1.342 

-  0.318 

4 .  Light  Bombers 

IL-28 

USSR 

-  0.439 

2.016 

5.  Medium  Bombers 

Tu-16  A/B 

USSR 

0.473 

3.737 

Notice  that  a  definite 

pattern  exists 

in  the  scores. 

Inter- 

ceptors  are  characterized  by  high  speeds  and  high  performance 
and  thus  have  high  defensive  scores.  Logically,  the  system 
with  the  highest  score  has  the  greatest  capability  —  in  this 
case,  the  British  Lightning.  Bombers  score  high  on  the 


^^Mihalka,  op.  cit. ,  p.  18. 
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offensive  dimension  because  of  their  typically  high  payload 
and  range.  Strike  aircraft  and  multipurpose  fighters 
generally  score  in  the  middle  ranges  of  both  dimensions. 

After  deriving  capability  scores  for  individual  systems, 
Mihalka  takes  the  analysis  one  step  further  and  combines 
them  with  information  on  the  numerical  inventories  of  a  sub¬ 
set  of  Middle  East  countries  to  obtain  capability  indices 
for  each  country's  weapons  stockpile.  The  precise  relation¬ 
ship  is  given  by 


C  =  F  *  I 

^i,k,t  i,k,s  ■^s,t 

where  £  denotes  a  particular  weapons  system,  t  the  time, 
i  the  class  of  system  (e.g.  air,  groiand,  naval)  ,  k  the 
factor,  F  the  factor  score,  ^  the  numerical  inventory,  and 
C  the  capability  score  of  the  inventory.  Mihalka 's  results 
are  provided  in  Table  III  for  illustrative  purposes  (see 
page  35) . 

At  first  glance,  using  factor  analysis  to  measure  capa¬ 
bility  seems  to  be  a  promising  technique  for  arms  transfer 
analysis.  For  example,  when  Mihalka  uses  capability  scores 
as  variables  in  a  linear  model  to  predict  the  military 

capability  of  several  Middle  East  countries,  the  explained 

2  27 

variance,  R  ,  reinged  from  0.65  to  0.93.  This  indicates  a 


27 


Ibid. , 


pp.  31-42. 
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TABLE  III' 


28 


DERIVED 

AIRCRAFT 

CAPABILITY  INVENTORIES  FOR 

MIDDLE 

EASTERN  SUBSET 

Year 

Irag 

Israel 

Jordan 

Syria 

Egypt 

49 

0.0 

0.0 

0.0 

0.0 

34.87 

50 

1.640 

0.02 

0.0 

0.0 

49.74 

51 

3.110 

0.04 

0.0 

0.0 

63.18 

52 

4.440 

0.06 

0.0 

0.0 

75.17 

53 

5.640 

25.64 

0.0 

4.380 

67.65 

54 

32.86 

23.09 

0.0 

8.320 

60.88 

55 

33.95 

151.2 

0.0 

37.42 

507.8 

56 

37.87 

305.9 

15.76 

207.5 

451.8 

57 

92.44 

275.3 

14.18 

191.1 

584.8 

58 

131.5 

247.8 

40.05 

172.0 

526.3 

59 

136.3 

223.1 

36.05 

154.8 

473.7 

60 

242.1 

343.8 

36.08 

150.1 

453.0 

61 

282.3 

309.4 

32.46 

145.8 

726.0 

62 

265.3 

278.5 

29.22 

142.0 

1100.0 

63 

244.9 

462.7 

26.31 

138.6 

1017 

64 

304.8 

416.4 

23.67 

135.5 

942.4 

65 

298.4 

394.3 

21.29 

121.9 

1013 

66 

401.6 

382.8 

19.18 

258.9 

970.0 

67 

501.2 

424.9 

25.62 

317.5 

1105 

68 

617.7 

468.5 

136.8 

285.7 

1049 

^^Ibid. ,  p.  22. 
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certain  degree  of  predictive  validity  since  the  closer  R 
is  to  1.0,  the  more  successful  the  model  is  at  predicting 
the  dependent  variable.  Another  positive  feature  is  the 
interval  measurement  of  capability,  not  only  because  of 
the  achieved  measurement  level  but  because  of  the  salience  of 
capability  as  a  unit  of  measure  as  well.  Successfully 
measuring  capability  would  provide  the  common  denominator 
needed  to  compare  weapon  systems  and  address  the  critical 
question  of  military  balance. 

However,  there  are  some  significant  problems  with  Mihal- 
ka's  methodology.  Consider,  first,  the  process  leading  to 
the  aircraft  inventory  capability  scores  (Table  III)  .  The 
first  step  of  this  calculation  involves  adjusting  the  derived 
weapons  capability  scores  (Table  II)  so  that  there  are  no 
negative  or  zero  values.  Mihalka  accomplishes  this  by 
adding  0.1  (selected  arbitrarily)  to  the  absolute  value  of 
the  lowest  factor  score  and  adding  the  resulting  sum  to 
each  aircraft  score.  The  effect  is  to  move  each  system  in 
a  positive  direction  along  the  interval  scale  by  the  same 
amount.  Recall  that  this  is  permissible  with  interval 
measurement  since  the  information  is  preserved  by  a  linear 
transformation.  Multiplying  these  adjusted  values  by  varying 
inventories  to  obtain  composite  country  scores  is  tenuous, 
however,  because  the  interval  nature  of  the  data  is  violated. 
An  example  will  illustrate  this.  Suppose  the  derived  factor 
score  for  aircraft  A  is  2.0  and  for  aircraft  B,  1.0,  along 
the  offensive  dimension.  If  a  country  had  an  inventory  of 
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25  A's,  the  country  capability  score  would  be  25  *  2.0  =  50.0. 
Similarly,  if  a  second  country  had  50  B's,  it's  capability 
score  would  also  be  50.0.  Clearly,  this  would  be  a  situation 
of  parity.  Now  consider  the  transformation  of  the  individual 
factor  scores  by  an  arbitrary  value  of  0.5,  i.e.,  aircraft 
A  =  2.5  and  aircraft  B  =  1.5.  Multiplying  these  adjusted 
capability  scores  by  the  same  coxintry  inventories  yields  a 
capability  score  of  67.5  for  the  first  (25  *  2.5  =  67.5), 
and  75.0  for  the  second  (50  *  1.5  =  75).  A  situation  of 
equality  has  suddenly  become  an  advantage  for  the  second 
country  without  any  change  in  the  nxomber  or  type  of  weapons. 
Since  Mihalka  used  the  country  capability  scores  to  deter¬ 
mine  his  residuals,  his  results  are  tenuous  and  may  not  be 
reliable.  Unless  Mihalka,  or  for  that  matter  any  analyst 
using  this  approach,  can  justify  a  varying  multiplicative 
transformation  on  interval  data,  this  will  always  be  the 
case. 

An  additional  problem  is  the  necessary  assumption  that 
the  selected  variables  adequately  describe  the  attribute 
being  measured  along  each  of  the  dimensions.  Many  analysts 
do  not  pay  enough  attention  to  variable  selection,  or  they 
justify  including  particular  variables  solely  on  the  basis 
of  high  factor  loadings.  .  Unfortxinately ,  even  a  very  high 
loading  (correlation)  between  factor  and  variable  does  not 
assure  proper  variable  selection.  The  high  nonsense  corre¬ 
lation  presented  by  British  statisticians  Yule  and  Kendall 
between  the  growing  number  of  radios  in  the  U.K.  and  the 
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growing  number  of  mental  defectives  with  correlation, 

2 

R  =  .99+,  and  explained  variance,  R  =  .99,  is  vivid  proof 
29 

of  this.  Thus,  unless  one  is  comfortable  with  the  asser¬ 
tion  that  speed,  technological  date,  and  perfoirmance  (which, 
incidentally,  is  not  defined  by  Mihalka)  equate  to  "defensive 
capability",  the  results  are  debatable. 

A  related  and  equally  important  concern  is  the  validity 
of  the  factors  themselves.  As  Gurr  points  out: 

Factor  analysis  results  always  pose 
problems  of  interpretation.  A  fundamental 
source  of  dispute  is  whether  the  factors 
are  merely  useful  artifacts  of  the 
analysis  or  whether  they  represent  latent 
but  real  phenomena. 

The  realist  position  predominates  at  the  present  time  and 
is  accepted  by  this  author.  Oftentimes  the  reluctance  of 
some  people  to  accept  the  validity  of  factors  can  be  traced 
to  poor  factor-labelling  more  than  the  actual  clustering  of 
variables  to  create  dimensions. 

A  second  attempt  to  measure  arms  in  terms  of  capability 
was  advanced  by  Lewis  Snider  in  1975.^^  Aside  from  a  dif¬ 
ference  in  labelling  of  the  two  factors  (air-to-air  combat 


29 

Tufte,  Edward  R. ,  Data  Analysis  for  Politics  and  Policy, 
p.  88,  Prentice  Hall,  1974. 

^^Gurr,  op .  cit. ,  p.  157. 

^^Snider,  Lewis,  Middle  East  Maelstrom.  The  Impact  of 
Global  and  Regional  Influences  on  the  Arab-Israeli  Conflict. 

1947-1973,  Ph.D.  Thesis,  University  of  Michigan,  1975. 
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and  groiind  attack  vs.  defensive  and  offensive) ,  the 
methodology  is  the  same  as  Mihalka's.  Snider,  however, 

derives  twelve  key  variables  instead  of  five  as  Mihalka 
32 

did.  The  clustering  of  these  variables  and  their  loadings 
are  reproduced  in  Table  IV. 

TABLE 

FACTOR  ANALYSIS  OF  PERFORMANCE 
CHARACTERISTICS  OF  COMBAT  AIRCRAFT 


Variable  Name 

Year  production 

Factor  I 

(Air-to-Air  Capability) 

Factor  II 
(Ground  Attack) 

began 

Speed :  primary 

.97 

.12 

mission 

.99 

.02 

Maximum  speed 

1.00 

.00 

Service  ceiling 
Horsepower  or 

1.00 

-.11 

Thrust 

.97 

.11 

Rate  of  climb 
Maximum  take-off 

1.00 

-.14 

weight 

.16 

.  95 

Ordnance  payload 

.16 

.95 

Ferry  range 

-.13 

1.00 

Combat  range 
Combat  radius : 

.00 

1.00 

external  fuel 
Combat  radius: 

.03 

.99 

internal  fuel 

-.20 

1.00 

32 

It  IS  assumed  that  Snider  and  Mihalka  both  started  with 
more  than  12  and  5  variables  respectively  although  the 
precise  numbers  are  unknown  by  the  author. 

^^Ibid. ,  p.  251. 
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This  is  a  more  comprehensive  selection  than  that  offered 
by  Mihalka,  but  it  is  still  open  to  criticism.  For  instance, 
thrust-to-weight  ratio  (T/W) ,  wing  loading  (W/S) ;  specific 
excess  power  (P^) ,  turn  rate,  and  turn  radius  are  the 
factors  which  are  used  to  describe  maneuverability  and 
aerial  combat  capability  in  authoritative  sources  such  as 
AGARD  Conference  Reports,  the  Navy  Tactical  Manual  for  F-4 
Aircraft,  the  Journal  of  Space/Aeronautics,  and  Aviation 
Week  and  Space  Technology.  Expressed  in  other  words,  aerial 
combat  capability  depends  on  both  the  potential  energy  of 
the  system  and  its  maneuverability.  It  is  clear  that 
Snider's  variables  relate  only  to  energy  factors  with  no 
regard  to  maneuverability  factors.  Hence,  the  systems  which 
score  highest  are  brute-force  aircraft  such  as  the  MIG-25 
FOXBAT,  while  less  powerful,  more  maneuverable  systems  such 
as  the  F-16  have  a  lower  scale  value.  However,  most  military 
commentators  would  not  accept  the  conclusion  that  the  F-16 
is  inferior  to  the  MIG-25  in  air-to-air  combat  capability. 
Again,  factor  analysis  demands  that  the  variables  define 
what  the  analyst  intends  to  measure  along  the  derived 
dimensions  as  precisely  and  completely  as  possible.  If  the 
analyst  cannot  be  sure  of  this ,  he  may  be  measuring  something 
other  than  what  is  intended. 

Although  Snider  does  not  calculate  country  capability 
scores,  he  does  calculate  a  composite  capability  score  which 
he  calls  the  "arms  transfer  score".  This  is  obtained  by 
multiplying  the  product  of  the  two  factor  scores  for  each 
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particular  weapon  by  the  number  of  weapons  transferred. 

Once  again  it  was  first  necessary  to  adjust  the  individual 
capability  scores  to  eliminate  zero  and  negative  values . 

In  contrast  to  Mihalka’s  conversion  formula,  Snider  adds  a 

value  large  enough  to  transform  the  highest  negative  score 
34 

to  1.00.  For  reasons  previously  discussed,  both  manipula¬ 
tions  have  a  profound  effect  on  the  scale  values  and  hence 
on  the  conclusions  and  results  of  Snider's  analysis. 

Hopefully,  the  inadequacies  of  the  three  measurement 
techniques  presented  in  this  chapter  have  been  clarified. 
Different  weaknesses  plague  each  one.  Dollar-value  measure¬ 
ment  is  perhaps  the  least  reliable  measure  of  arms  flow  and 
military  potential,  and  yet,  ironically,  is  the  approach 
most  often  used  in  arms  research.  Often  it  cannot  provide 
reliable  information  for  even  the  most  fundamental  questions 
such  as  determining  the  magnitude  of  arms  flow  —  due  to 
fluctuations  in  the  value  of  the  dollar.  Additionally,  it 
is  susceptible  to  differential  exchange  rates,  over/under¬ 
valued  exchanges  due  to  ancillary  conditions,  and  is  further 
constrained  by  its  inability  to  reflect  the  impact  of  arms 
in  relative  terms.  Numerical/inventory  measurement,  on  the 
other  hand,  provides  fairly  reliable  ratio  measurement  of 
arms  volumes  and  trends  and  has  been  used  successfully  for 
descriptive  analysis.  Yet  it  is  misleading  when  used  to 


^^Ibid. ,  p.  254. 
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represent  military  balance  or  capability.  Moreover,  it 
cannot  be  used  to  compare  the  relative  strengths  of  weapon 
systems.  Finally,  while  capability  measurement  using  factor 
analysis  does  provide  a  basis  for  comparing  weapons  and 
assessing  regional  military  balance,  it  breaks  down  in  cer¬ 
tain  applications,  most  notably  when  composite  capability 
scores  for  a  coimtry  or  groups  of  weapons  are  derived  from 
individual  weapon  scores .  There  are  also  uncertainties  that 
must  be  faced  in  selecting  (or  omitting)  crucial  variables 
and  in  interpreting  the  factor-dimensions  themselves . 

As  indicated  previously,  each  of  the  three  measurement 
techniques  considered  has  utility  in  certain  circiomstances . 
However,  for  studies  of  nations  and  regions  where  military 
factors  and  questions  of  military  balance  are  of  overwhelming 
importance,  such  as  the  Middle  East,  capability  measurement 
is  potentially  the  most  promising  of  the  three.  It  is  impor¬ 
tant  for  other  reasons  as  well.  For  one  thing,  intelligence 
analysts  are  always  faced  with  the  problem  of  assessing  an 
antagonist's  strengths  and  weaknesses.  Capability  measure¬ 
ment,  if  done  meaningfully,  could  provide  valuable  inputs 
to  improve  such  estimates.  Secondly,  such  an  evaluative 
technique  could  be  used  to  evaluate  U.S.  weaponry,  not  only 
by  facilitating  side-by-side  comparisons  with  Soviet  equip¬ 
ment,  but  also  through  refining  the  selection  and  evaluation 
of  new  systems.  Finally,  it  could  assist  the  policymaker 
by  providing  an  authoritative  input  into  the  arms  transfer 
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decision  process.  Knowing  the  relative  strength  of  U.S. 
systems  would  provide  some  guidance  as  to  the  size  and  type 
of  transfer  required  in  a  particular  situation. 

In  view  of  the  potential  value  of  capability  measure¬ 
ment  to  both  arms  transfer  studies  and  military  intelligence 
estimates,  the  remainder  of  the  thesis  will  concentrate  on 
several  possible  approaches  to  the  problem.  Successful 
measurement  of  capability  depends,  in  the  most  fundamental 
sense,  on  operationalizing  (defining)  capability  in  a  meaning¬ 
ful  way.  Is  it  an  attribute  defined  only  by  the  performance 
characteristics  of  the  weapon,  or  is  it  area-dependent  and 
affected  by  external  factors?  The  next  chapter  will  explore 
this  issue  in  more  detail. 
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III.  DEFINING  CAPABILITY  IN  FIGHTER  AIRCR?^T 


The  analyst  can  view  weapons  capability  in  one  of  two 
possible  ways:  as  an  intrinsic  attribute  of  the  particular 
weapon  expressed  through  the  performance  characteristics  of 
that  weapon;  or  as  a  more  complex  phenomenon  composed  of  ^ 
both  intrinsic  performance  characteristics  and  external 
factors  like  the  operating  environment,  operator  proficiency, 
and  the  technological  capacity  of  the  user.  The  first 
approach  can  be  seen  in  the  factor  analytic  method  employed 
by  Snider  and  Mihalka  where  only  performance  variables  are 
combined  to  obtain  system  capability  scores.  Among  other 
things,  the  most  significant  ass;imption  behind  such  an 
approach  is  that  system  capability  is  invariant  and  that  addi¬ 
tional  forms  which  might  effect  a  weapon's  capability  in  a 
given  situation  heed  not  be  considered.  The  second  approach 
has  not  been  applied  to  capability  analysis  per  se  in  any 
study  familiar  to  this  author.  Geoffrey  Kemp's  work  on 
classifying  weapons  systems  and  force  designs  comes  closest 
to  incorporating  this  multi-faceted  view  inasmuch  as  he 

considers  environmental  and  regional  factors  to  have  a  direct 

35 

impact  on  weapons  effectiveness.  Obviously,  allegiance  to 
this  view  means  that  capability  measurement  would  be  area 


^^See  especially,  Kemp,  Geoffrey,  Classification  of 
Weapons  Systems  and  Force  Designs  in  Less  Developed  Country 

Environments,  C/70-3,  MIT,  1970 . 
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dependent,  with  a  system's  relative  potential  fluctuating 
according  to  outside  influences. 

With  these  comments  in  mind,  the  main  objective  of  this 
chapter  is  to  derive  definitions  for  the  aerial  combat  capa¬ 
bility  of  fighter  aircraft  using  both  approaches.  An  analy¬ 
sis  of  pertinent  aerodynamic  equations,  along  with  weapons 
effectiveness  studies  by  Kemp  and  others  is  performed  to 
identify  the  required  variables.  In  the  process,  the  com¬ 
plexity  of  capability  analysis  is  demonstrated  and  solutions 
to  some  data  acquisition  problems  provided.  No  attempt  is 
made  to  argue  in  favor  of  either  of  the  philosophical  views. 
Rather,  it  is  hoped  that  the  strengths  and  weaknesses  of  each 
are  brought  to  light,  particularly  as  they  relate  to  scaling 
and  measurement. 

Recall  that  both  Snider  and  Mihalka  used  the  same  three 
criteria  to  determine  the  relevance  of  variables.  With  minor 
modifications,  they  are:  (1)  the  ability  to  describe  the 
system  meaningfully;  (2)  ease  of  quantification;  and  (3)  the 
availability  of  data.  All  are  sound,  realistic  principles 
and  serve  the  variable  selection  process  quite  nicely.  The 
most  constraining  of  the  three  is,  of  course,  data  availa¬ 
bility.  It  is  an  unfortunate  (although  sometimes  necessary) 
fact  that  much  of  the  important  information  needed  for  capa¬ 
bility  assessment  is  classified  and  thus  unavailable  to  the 
research  community  at  large.  Even  more  unfortunate  is  the 
compromising  effect  such  information  gaps  have  on  what  should 
be  the  most  important  requirement  —  a  meaningful  representation 


45 


of  the  system.  For  aircraft  this  means,  at  a  minimum, con¬ 
sidering  the  aerodynamic  qualities  of  the  airframe,  the 
weapons  suit  typically  employed,  and  its  endurance  (range) . 

Authoritative  sources  such  as  the  Navy ' s  Tactical  Manual 
for  the  F-4,  and  AGARD  (Advisory  Group  for  Aerospace  Research 
and  Development)  Conference  Reports,  describe  the  aero¬ 
dynamic  requirements  of  fighter  aircraft  in  terms  of  two 
qualities  —  energy  and  maneuverability.^^  A  plausible  way 
to  uncover  meaningful  variables,  therefore,  is  to  consider 
the  aerodynamic  equations  describing  these  phenomena.  First, 
with  respect  to  energy,  the  total  energy  of  a  system  is  the 
sum  of  its  potefitial  and  kinetic  energy.  For  an  aircraft  in 
flight  this  is  represented  by: 

E  =  mgh  +  j  mv^  (1) 


where  m  is  the  mass  of  the  aircraft,  £  is  the  force  of 
gravity,  h  is  the  altitude,  and  v  is  the  aircraft's  velocity. 
Frequently,  system  energy  is  expressed  as  specific  energy. 

Eg,  which  is  simply  Equation  (1)  divided  by  the  system 
weight,  W.  That  is. 


2g 


(2) 


^^See  especially,  AGARD  Conference  Proceedings  No.  62, 
Preliminary  Design  Aspects  of  Military  Aircraft,  Harford 
House,  March  1970,  and  McDonnell  Douglas '  Tactical  Manual 
Navy  Model  F-4B  and  F-4J  Aircraft  (U) ,  NAVAIR  01-245  FDB-lT, 
pp.  1-2  -  1-3,  1  March  1971. 
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Whatever  expression  is  used,  it  is  clear  that  the  key 
variables  are  altitude,  h,  and  velocity,  v.  Increasing 
altitude  and  holding  velocity  constant,  for  instance,  would 
produce  an  increase  in  total  system  energy.  A  similar 
increase  in  system  energy  results  when  velocity  rises. 
Understanding  this  simple  relationship  is  important  since 
aerial  combat  is  largely  concerned  with  maintaining  as  high 
an  energy  level  as  possible.  Given  this  maxim,  the  importance 
of  speed  and  effective  combat  ceiling  relative  to  capability 
is  legitimized. 

In  addition  to  speed,  the  air  superiority  system  must 
possess  excellent  acceleration  capability  within  its  flight 
envelope.  In  level  flight,  the  available  logitudinal 
acceleration,  A^,  is  given  by. 


=  g(T-D) 
"x  W 


(3) 


where  £  once  again  is  gravitational  force,  T  is  system  thrust, 
D  is  system  drag  (or  resistance) ,  and  W  system  weight.  Note 
that  the  ability  to  accelerate  longitudinally  does  not 
involve  velocity  at  all,  depending  instead  on  thrust,  drag, 
and  weight.  This  suggests  that  relying  on  velocity  alone  to 
represent  fighter  capability  is  inadequate.  It  should  be 
clear  that  the  greater  the  ratio  — ^ —  ,  the  greater  the 
system's  ability  to  accelerate.  Much  of  the  U.S.  emphasis 
in  smaller,  light-weight,  aerodynamically  clean,  fighter 
designs  (e.g.,  the  F-16,  F-17,  and  F-18)  reflects  a  desire 
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to  improve  present  capability  in  this  area,  and  substan¬ 
tiates  including  the  thrust  -  drag  -  weight  relationship 

in  the  variable  list  describing  fighter  capability. 

T  —  D 

Both  velocity  and  the  — ^ —  ratio  are  brought  together 
in  the  expression  for  specific  excess  power,  P^,  a  quantity 
which  can  be  thought  of  as  the  system's  ability  to  change 
energy  levels.  (It  also  serves  as  the  expression  for  the 
rate  of  climb.)  More  precisely. 


(with  all  symbols  defined  previously) .  This  is  the  ideal 
expression  to  use  for  comparing  aircraft  energy  capabilities 
since  it  accounts  simultaneously  for  velocity  and  accelera¬ 
tion.  (U.S.  fighter  pilots,  in  fact,  use  data  to 
determine  areas  of  definite  energy  advantage  and  disadvantage 
for  their  own  aircraft  in  relation  to  major  opposition 
platforms.)  Unfortunately,  P^  data  for  most  combat  systems 
are  difficult  to  find  in  open  sources,  and  in  some  cases 
even  remain  incomplete  in  classified  literatTire.  The  main 
problem,  in  most  cases,  is  accurately  determining  the 
magnitude  of  drag,  D,  at  various  altitudes,  airspeeds  and 
configurations.  Two  solutions  are  possible:  (1)  eliminate 
Pg  comparisons  and  use  thrust- to- weight  ratio  (T/W)  and 
velocity  as  two  separate  variables;  or  (2)  estimate  the 
aircraft's  drag  using  size,  shape,  and  wing  platform 
information.  The  first  solution  is  expeditious  and,  for 
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the  most  part,  very  adequate.  However,  the  fact  that 
estimates  of  aircraft  drag  can  be  made  is  significant 
because  it  suggests  that  determinations  are  possible, 
and  further  that  data  availability  problems  for  air  combat 
systems  are  less  severe  than  they  initially  appear  to  be. 

No  less  significant  than  the  energy  state  of  a  fighter 
aircraft  is  its  maneuverability.  Maneuverability  can  be 
viewed  as  the  ability  to  achieve  a  high  rate  of  turn  and 
a  small  radius  of  turn  throughout  the  flight  envelope.  Rate 
of  turn,  0^,  is  defined  as; 


e, 


(5) 


T 


V 


where  is  the  acceleration  normal  to  the  flight  path,  £  is 
gravitational  force,  and  V  is  velocity.  As  the  equation 
shows,  for  a  given  combat  speed,  the  rate  of  turn,  9^, 
depends  entirely  on  the  acceleration  normal  to  the  flight 
path,  A^ .  Redefining  A^  and  sxibstituting  the  new  expression 
into  the  turn  rate  equation  (5)  reveals  several  additional 
variables : 


A 


C, 


(6) 


thus 


(7) 


49 


such  that  £  is  dynamic  pressure  caused  by  the  velocity  of 
the  aircraft  through  a  certain  air  density,  ,  is  the 
lift  coefficient,  W/S  is  the  wing  loading,  the  ratio  of 
aircraft  weight  to  wing  area,  L  is  the  lift  generated  by 
the  system,  W  is  the  system  weight,  and  n  the  load  factor. 

In  other  words,  to  produce  high  normal  acceleration,  A^,  and 
a  high  turn  rate,  the  system  must  be  able  to  generate  lift 
efficiently  in  relation  to  its  weight.  Knowing  the 
particular  load  factor  values  (n)  for  two  aircraft  under 
comparative  conditions  would  provide  this  information. 
However,  these  values  are  not  normally  available  in 
unclassified  sources  because  lift  data  can  only  be  derived 
from  a  technical  analysis  of  the  aircraft  and  as  such 
usually  goes  beyond  the  scope  and  purpose  of  open  source 
material.  However,  as  equation  (6)  reveals,  a  reasonable 
indication  of  this  capacity  is  given  by  the  wing  loading, 

W/S,  since  the  lower  the  wing  loading  at  a  given  velocity, 
the  higher  the  lift-to-weight  ratio  (L/W)  and,  hence, 
maneuverability.  Similarly,  if  W/S  is  relatively  high, 

L/W  will  be  relatively  low  as  will  the  maneuverability  of 
the  system.  This  relationship  justifies  including  the  W/S 
ratio  on  the  variable  list  defining  capability  as  a  surrogate 
for  the  turning  rate,  9,^,. 

If  the  turning  rate  is  known,  a  simple  conversion 
procedure  gives  the  radius  of  turn,  R. 


g/  l/w  -  1 
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Since  no  new  variables  are  introduced,  it  is  actually 
redundant  to  be  concerned  with  the  turn  radius  when  the 
turn  rate  is  known.  Hence,  it  is  sufficient  to  use  just 
one  of  these  expressions  to  help  define  maneuverability. 

On  the  strength  of  studies  done  by  the  Northrop  Aircraft 
Corporation  which  suggest  that  turning  rate  is  a  more 
critical  factor  in  air  combat  than  turning  radius, 
turning  rate  is  preferred  by  the  author. 

As  is  the  case  with  specific  excess  power  data,  obtaining 
pre-calculated  turn  rates  and  radii  for  a  large  sample  of 
aircraft  requires  access  to  classified  sources.  However, 
reasonable  estimates  for  the  turn  rate  and  turn  radius  can 
be  made,  again  from  dimensional  information  on  the  aircraft. 
(The  interested  reader  is  referred  to  A.W.  Babister's 
Aircraft  Stability  and  Control  for  examples  of  such  calcula¬ 
tions.)  Again,  it  is  important  to  note  that  information 
about  key  variables  can  be  obtained  using  open  sources  if 
the  researcher  is  willing  to  exploit  technological  informa¬ 
tion  and  theory. 

To  review  quickly,  analyzing  the  aerodynamic  expressions 
for  energy  and  maneuverability  identified  some  eight  important 
variables  which  are  directly  related  to  the  aerodynamic 
capability  of  a  fighter  aircraft:  velocity  (V) ,  acceleration 
as  indicated  by  the  thrust-to-weight  ratio  (T/W) ,  specific 
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p.  47. 
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excess  power  (Pg) /  wing  loading  (W/S) ,  turn  rate  (9^) , 
turn  radius  (R) ,  the  load  factor  (L/W  =  n) ,  and  the 
coefficient  of  lift  (C^^)  .  Because  of  the  existing  inter¬ 
relationships  between  them,  however,  the  list  can  be 
distilled  down  to  four  items:  velocity;  thrust-to-weight 
ratio;  wing  loading;  and  turn  rate.  In  reality,  these  four 
variables  reflect  the  author's  preference  and  by  no  means 
constitutes  the  only  plausible  selection.  What  is  important 
is  not  the  variables  themselves  but  the  fact  that  both 
energy  and  maneuverability  qualities  are  represented.  Any 
capability  analysis  of  air  superiority  aircraft  which  does 
not  accomplish  this  is  incomplete. 

Along  with  the  aerodynamics  of  the  platform,  of  major 
importance  to  aerial  combat  capability  is  the  system's 
endurance.  The  swiftest,  most  maneuverable  fighter  in  the 
world  would  be  of  little  value  without  the  necessary  endurance 
to  perform  the  mission.  Some  Soviet  fighters,  and  the  MiG-21 
in  particular,  suffer  in  this  respect.  Measuring  this 
characteristic  poses  no  problem  since  any  one  of  a  number  of 
range  figures  routinely  appears  in  open  sources  including 
ferry  range,  combat  range,  combat  radius,  and  specific  fuel 
consumption  (a  measure  of  the  amount  of  fuel  burned  per  hour) . 
The  one  preferred  here  is  combat  radius  since  it  is 
calculated  on  the  basis  of  the  aircraft's  primary  mission 
and  thus  captures  the  essence  of  realistic  employment  of 
the  aircraft. 
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The  third  and  final  aspect  to  consider  is  the  basic 
capability  of  the  weapons  suit  normally  employed  with  the 
aircraft.  For  air  combat  systems  this  means  either  light 
automatic  cannons,  air-to-air  missiles,  or  both.  Because 
of  the  great  similarity  between  aircraft  g\in  systems 
throughout  the  world,  it  is  do\ibtful  that  a  side-by-side 
comparison  of  such  systems  would  reveal  much  that  would 
influence  or  alter  the  relative  capabilities  of  two  gun- 
equipped  fighters.  For  the  most  part,  the  guns  are  20-23  mm 
in  size,  having  firing  rates  of  about  1000  rounds  per  minute, 
and  effective  ranges  of  less  than  one  mile.  The  impact  of 
minor  variations  in  these  specifications  would  be  difficult, 
if  not  impossible,  to  express  in  the  capability  measure  of 
the  entire  fighter  weapon  system  (i.e.,  the  combined  capability 
of  the  airframe's  aerodynamics,  the  weapons  suit,  and  mission 
range  capacity) .  On  the  other  hand,  the  presence  or  absence 
of  a  gain  system  should  be  considered  in  the  assessment  since 
the  gun-equipped  fighter  has  more  destructive  potential  in 
an  aerial  combat  situation  than  its  giinless  counterpart 
(assuming,  of  course,  that  the  gun's  added  weight  does  not 
hinder  maneuverability,  etc.) 

A  bit  more  attention  must  be  given  to  individual  differ¬ 
ences  in  missile  systems  since  they  are  more  pronomced  and 
could  make  a  difference  in  the  fighter's  assessed  capability. 
Perhaps  the  best  single  indicator  of  a  missile  system's 
capability  is  its  launch  envelope  which  tells  something 
about  missile  range,  maneuverability,  and  required  launch 
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parameters.  Again,  finding  this  information  in  open  sources 
represents  a  formidable  problem.  Because  of  this,  other, 
more  general,  factors  must  be  considered  to  piece  together 
the  missile's  true  capability. 

One  such  factor  is  the  guidance  system.  Differences  in 
guidance  schemes  account  for  differences  in  firing  envelopes 
and  tactics.  A  beam-rider  such  as  the  Soviet-built  Alkali 
or  a  semi-active  homing  system  such  as  the  AIM-7  built  by 
Raytheon,  for  instance,  can  be  fired  at  a  closing  target, 
or  one  that  is  crossing  the  flight  path.  This  means  that 
the  pilot  does  not  have  to  achieve  a  position  behind  the 
target  aircraft  to  fire  and,  further,  that  the  aircraft's 
maneuverability  is  not  necessarily  critical  to  a  successful 
attack.  In  contrast,  missiles  relying  on  electro-optical 
guidance,  e.g.,  heat-seekers  such  as  the  Atoll  and  Side¬ 
winder,  must  be  fired  at  the  rear  quadrant/exhaust  area  of 
the  opposing  aircraft  and  thus  are  totally  dependent  upon 
the  aircraft's  capability  to  maneuver  into  an  advantageous 
position.  This  suggests  that  mating  a  particular  missile 
with  an  aircraft  can  either  place  special  demands  on  the 
aircraft  or  compensate  for  its  deficiencies.  Theoretically, 
for  example,  an  A-6  attack  aircraft,  or  even  an  S-3,  could 
be  used  in  an  air-superiority  role  if  equipped  with  a  long- 
range  missile  like  the  Phoenix,  because  the  missile  does 
all  the  work  and  demands  nothing  of  the  aircraft  except 
target  acquisition,  and  transportation  to  the  launch 
position. 
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The  previous  example  suggests  a  second  general  charac¬ 
teristic  that  can  be  used  to  gauge  missile  capability  —  range 
Having  a  missile  that  can  reach  a  target  more  than  50  miles 
away  certainly  is  more  formidable  than  one  with  a  2-3  mile 
range,  and  the  difference  should  be  reflected  in  the  system's 
assessed  capability. 

Finally,  inasmuch  as  missiles  depend  on  the  same  aero¬ 
dynamic  qualities  as  aircraft,  knowing  something  about  their 
maximum  speeds  and  maneuverability  would  contribute  to  the 
task  of  determining  how  much  capability  they  added  to 
individual  aircraft  systems,  especially  in  an  aerial  combat 
environment.  A  figher  equipped  with  a  highly  maneuverable 
missile  similar  to  the  one  depicted  in  Figure  1  (p.  56)  would 
have  a  decided  edge  over  any  missile  system  currently  in 
operation  and  enhance  the  probability  of  destroying  an 
opponent  because  of  superior  aerodynamic  qualities. 

Despite  the  painstaking  process  used  to  isolate  salient 
features  of  the  air  superiority  system  in  terns  of  its  aero¬ 
dynamic  qualities,  range,  and  weapons  suit,  technological 
advances  and/or  design  innovations  which  affect  performance 
and  capability  cannot  be  captured  with  the  variables  selected 
To  illustrate,  one  of  the  newest  U.S.  fighters,  the  F-16,  has 
fuselage  strakes  (Figure  2,  p.  57)  which  produce  vortices 
that  improve  lift  and  make  the  effective  wing  area 
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FIGURE  1 


ENHANCED  CAPABILITY  WITH  HIGHLY 
MANEUVERABLE  MISSILE 
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FIGURE  2 


ADVANCED  TECHNOLOGY  OF  F-16 
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significantly  greater  than  the  actual  geometric  area. 

The  calculated  wing  loading,  W/S,  is  thus  higher  than  the 
actual  wing  loading.  It  follows  that  using  the  calculated 
wing  loading  to  indicate  its  maneuverability  would  lead  to 
an  underestimation  of  the  F-16's  maneuvering  capability. 

To  account  for  such  occurrences  it  is  recommended  that  the 
year  of  production,  or  some  other  time-related  reference, 
be  included  in  the  variable  list.  Both  Snider  and  Mihalka 
found  this  to  be  useful  in  their  assessments  of  aircraft 
capability. 

To  review,  using  open  source  information  and  aerodynamic 
formulae,  the  following  variables  have  been  selected  to 
define  the  aerial  combat  capability  of  fighter  aircraft: 

I .  Energy /Maneuverability 

1.  Velocity 

2.  Thrust-to-weight  ratio  (T/W) 

3.  Wing  loading  (W/S) 

4.  Turn  rate,  0^ 

II.  Endurance 

5 .  Combat  radius 

III.  Weapons  Configuration 

6.  Guns  —  presence  or  absence 

7.  Missiles  —  operationalized  by  guidance, 
velocity,  range  and  maneuverability  parameters 

IV.  Technological  Innovation 

8.  Year  of  Production 
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A  brief  but  useful  discussion  on  the  effect  of  strakes 
on  fighter  maneuverability  can  be  found  in  AGARD  Conference 
Proceeding  No.  102,  pp.  24-5,  and  24-6,  Harford  House,  1972. 
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For  many  in  the  intelligence  community,  the  technical 
analysis  presented  in  this  chapter  is  familiar.  Pilots  and 
aeronautical  engineers  would  also  feel  comfortable  thinking 
in  terms  of  the  variables  presented.  For  others,  particularly 
those  interested  in  using  capability  to  determine  the 
political/military  consequences  of  arms  transfers,  such 
complexity  may  seem  unnecessary.  The  author  is  convinced, 
however,  that  without  a  sufficient  amount  of  probing  and 
technical  analysis,  erroneous  or  misleading  capability 
assessment  is  highly  probable.  Recall  the  variable  list 
Snider  used  to  compute  aircraft  capability  scores  (Table  IV, 
p.  39) .  They  provide  an  excellent  description  of  the  energy 
dimension  of  the  aircraft  considered,  but  completely  ignore 
maneuverability  qualities.  The  result  is  to  accentuate  the 
capability  of  powerful,  fast  aircraft  (i.e.  pure  interceptors) 
vis-a-vis  less  powerful  but  more  maneuverable  air  superiority 
systems,  thereby  misrepresenting  the  real  capability  of  air 
superiority  aircraft.  This  explains  the  "curious"  location 

of  the  F-14  on  Snider's  graphical  display  of  aircraft 
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capability  (Figure  3,  p.  60).  According  to  Snider's 
analysis,  the  F-14A  is  inferior  to  the  MiG-23,  MiG-25,  F-4E, 
and  F-15A  as  an  air  superiority  platform.  In  reality,  it  is 
inferior  only  in  terms  of  engine  thrust  and  speed,  not  in 
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Snider's  results  would  better  sort  themselves  out  if 
instead  of  a  single  dimension  for  interceptors  and  air 
superiority  systems,  separate  indices  were  constructed  for 
each  aircraft  type. 
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FIGURE  3 
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terms  of  maneuverability,  and  certainly  not  in  terms  of 
its  weapons  suit.  The  dangers  and  pitfalls  of  oversimplifica¬ 
tion  are  apparent  in  this  particular  instance. 

Some  people  argue  that  weapons  capability  is  much  more 
than  a  combination  of  performance  characteristics.  With 
respect  to  aerial  combat  capability,  for  example,  there  is 
empirical  evidence  which  suggests  that  pilot/crew  proficiency 
is  the  ultimate  determinant  of  fighter  capability.  The  Navy's 
fighter  community  can  trace  much  of  its  combat  success  during 
the  latter  stages  of  the  Vietnam  War  to  better  training  and 
crew  proficiency.  Prior  to  1968,  the  Navy's  kill  ratio 
against  the  North  Vietnamese  was  a  rather  unimpressive 
2.9  to  1.  After  the  establishment  of  the  Fighter  Weapons 

School,  the  figures  steadily  improved  until,  by  1972,  it  had 
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reached  12  to  1.  Another  pertinent  example  is  found  in 
the  Middle  East  where  the  superior  training  and  tactics  of 
Israeli  pilots  have  paid  huge  dividends  in  maintaining  air 
superiority  against  countries  equipped  with  some  of  the  best 
aircraft  in  the  world. 

The  importance  of  operator  proficiency  can  be  generalized 

to  other  weapons  and  combat  milieu.  Quoting  Kemp; 

...the  superiority  of  Israel's  armed  forces 
in  terms  of  their  ability  to  use  weapons 
effectively  has  meant  in  the  past  that  the 
qualitative  difference  between  Egypt's 
modern  T-55  tanks  and  Israel's  less  modern 
Centurions  has  not  been  so  important  as  would 
have  been  the  case  if  both  sides  were  equally 
proficient  at  fighting  with  these  weapons. 41 

^ ^Aviation  Week  and  Space  Technology,  p.  62,  3  December  1973. 

^^Kemp,  Geoffrey,  Classification  of  Weapons  Systems  and 
Force  Designs  in  Less  Developed  Country  Environments  C/70-3, 
p.  32,  MIT,  1970. 
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Kemp's  work  on  weapons  classification  and  force  design 
isolates  other  factors  which  impact  on  weapon  system 
potential.  Mainly,  he  argues  that  the  only  realistic  way 
to  assess  a  weapon  or  military  force's  potential  quality 
is  to  set  the  technological  qualities  of  the  system  or  force 
against  the  unique  environmental  factors  of  the  operating 

area.  Of  the  six  specific  environmental  characteristics 
42 

Kemp  feels  are  significant,  three  in  particular  can  be 
linked  to  capability  assessment: 

(1)  the  geography  of  the  area 

(2)  the  combat  environment,  that  is,  the  technical 
competence,  fighting  ability,  and  force  levels 
or  the  adversary 

(3)  the  technical  competence  and  fighting  ability  of  the 
force  using  the  weapon. 

Geography  is  most  applicable  to  ground  system  capability 
assessment  since  air  and  naval  systems  have  stable  operating 
environments  world-wide .  One  example  showing  geographic 
effects  on  ground  system  capability  is  provided  by  Snider 
who  points  out  that  the  intrinsic  amphibious  capability  of 
Soviet  tanks  is  tremendously  important  to  the  Egyptians 
because  of  the  Suez  Canal  but  is  hardly  relevant  in  countries 


42 

Kemp,  Geoffrey,  "Arms  Traffic  and  Third  World  Conflicts, 
International  Conciliation,  No.  577,  p.  25,  March  1970. 
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4  3 

like  Algeria  and  Afghanistan.  A  more  graphic  example  is 

provided  by  the  Insight  Team  of  the  London  Siinday  Times. 

44 

As  Figure  4  depicts,  Soviet  tanks  exhibit  low  profiles 
and  little  gun  depression  capability,  and  as  such,  are 
optimized  for  flat  terrain.  In  contrast,  U.S.  and  British 
tanks  have  larger  turrets  and  greater  gun  depression 
capability  which  make  them  better  suited  for  the  ^Indulating 
terrain  of  the  Middle  East.  When  geography  optimizes  or 
degrades  a  weapon's  performance  in  this  fashion,  corresponding 
adjustments  in  the  weapon's  assessed  capability  must  be  made 
in  order  to  bring  a  certain  degree  of  reality  to  the 
assessment. 

According  to  Kemp's  findings,  the  utility  of  any  weapon 
system  is  determined  largely  by  the  combat  environment. 

As  Tables  V  through  VII  (pp.  65-67)  show,  different  aircraft 
are  preferred  under  different  circxamstances  because  the 
mission  requirements  vary.  Observe,  for  instance,  that  high 
speed  capability  is  important  in  a  hostile  environment  but 
becomes  unnecessary  in  permissive  situations.  Even  within 
the  hostile  environments  categorized,  the  relative  importance 
of  speed  varies.  Inasmuch  as  utility  is  a  measure  of 


43 

Snider/  Lewis  W. ,  Arabesque;  Untangling  the  Patterns 
of  Supply  of  Conventional  Arms  to  Israel  and  the  Arab  States 

and  Their  Effects  on  the  Arab/Israeli  Conflict  1948-1973, 

paper  presented  at  the  1975  Annual  Convention  of  the  Inter- 

national  Studies  Association,  Washington,  D.C.,  19-22 
February  1975,  p.  10. 

44 

Insight  Team  of  the  London  Sunday  Times.  The  Yom 
Kippur  War.  Garden  City,  N.Y.,  Doubleday,  1974. 


63 


T^kjfkinfl  iniTuiU  doyvn'posilioa 


FIGURE  4 
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TABLE  V 


PERMISSIVE  COIN 


ISOLATED  AlH  STRIKE  TACTICAL  SUPPORT 


M.1l3slon  1 

Mission  2 

Ist 

2  Squadronu  B-26K 

Jst 

1  ^/2  Squadrons  B-57 

PreCerreU 

2nd 

1  ^/2  Squadrons  I1-37 

2nd 

1  Squadron  A-37B 

Syatvae 

3rd 

L  Squadron  A-4K 

3rd 

2  Squadrons  3-2bK 

4  til 

2  Squadrons  A- 3711 

4  til 

1  Squadron  A-4F 

let 

High  Total  Payload 

1st 

High  Total  Payload 

Charaecerleclca 

2na 

Multiple  Armament 

2nd 

Multiple  Armament 

liVf  luaucing 
S^ecClon  of 

3rd 

Good  Loiter  Capability 

3rd 

Cood  Loiter  Capability 

iac  Cwtce 

4  til 

Combat  Well  Tested 

4  th 

Coud  Low  Altitude  Performance 

-  HOSTILE 

COIN 

ISOLATED  AIR  STRIKE 

TACTICAL  SUPPORT 

Mission  5 

Mission  b 

1st 

1  Squadron  A-4K 

1-tl 

L  Squadron  A-4F 

Preferred 

2iid 

.  ^/2  Detacliment  F-4C 

2nd: 

i  ^/2  Squadrons  B-37 

S'yaCeaiM 

3rd 

1  ^/2  Squadrons  B-57 

3rd 

2  Squadrons  A-37B 

4  til 

1  Squadron  F-5A 

4  til 

^/2  Detachment  F-4C 

1 

let 

High  Total  Payload 

1st 

High  Total  Payload 

Character  laciCM' 

2nd 

Hlgli  Speed  Capability 

2nd 

Multiple  Armament 

Influencing 
Selection  of 

3rd 

Multiple  Armament 

3rd 

Cood  Loiter  Capability 

lac  Choice 

4  til 

High  Combat  Radius 

4  til 

High  Speed  Capability 

Package  (I  Squaiirnn  •  approx.  12  aircraft; 

1  Decactimenc  ~  approximately  6  aircraft} 


- 

2  SQUADRONS  A-37B 

1  SQUADRON  F-5A 

Choices  of 

1  1/2  SQUADRONS  CANBERRA  (8-57) 

Systems  Offered  ' 

■  1  SQUADRON  A-4K 

for  Ail 

1  SQUADRON  SU-7 

Attack 

2  SQUADRONS  OV-lOA  (BRONCO) 

Missions 

1  SQUADRON  MrC-21D 

2  SQUADRONS  B-26K 

1  1/2  SQUADRONS  HUNTER  OR  F-86 

- 

1  DETAOiHENT  LIGHTNING 

1  DETACHMENT  MIRAGE  III 

' 

1/2  DETACHMENT  F-4C 
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TABLE  VI 


PERMISSIVE 

EXTERNAL 

ISOLATED  AIR  STRIKE 

TAC  SUPPORT 

MISSION  3 

MISSION  4 

IsC. 

1  1/2  Sqaadrona  B-57 

Ut 

1  1/2  Squadrons  B-57  ! 

Prifftfrrtfvl 

2nd 

1  Squadron*  B-26K 

1  S<]uadron  A-4F 

Svscusit 

3rd 

1  Squadron  A-4P 

3rd 

2  Squadrons  B-26K 

Slh 

2  Squadrons  A-37B 

4ch 

Z  Squadrons  A-37R 

lac 

Hljth  Total  Payload 

Isc 

Hlr.h  Total  Pavload 

Characturlscii 

‘  2nd 

Mul  1  le  A 

2nd 

M  1  1  1  A  asmn 

InflueiiciiiK 

ru  p  e  rmamen 

.  u  t  p  e  mament 

Selection'  of 

3rd 

Good  Loiter  Capability 

3rd 

rase-tum  Round  Time 

IsC  Choice 

6ch 

Good  Low  Alclcuda  Performance 

4ch 

Good  Low  Altitude  Performance 

Ease  of  Maintenance 

1 

HOSTILE 

EXTERNAL 

i 

'  TCCII  4T1.T>  ATB  CTSTlfP 

TAC  SUPPORT  ! 

1 

MISSION  7 

MISSION  8 

1  Preferred  • 

isc 

I  Squadron  A-4  F 

lac 

1  Squadron  A-4F 

2nd 

1/2  Detachment  P-4C 

2nd 

1/2  Detachment  r-4C 

3rd 

1  Detachment  Mirage  HI 

3rd 

1- Detachment  Mlrajte  III 

4ch 

1  Squadron  F-5A 

4  til 

^  Squadron  F-SA 

Isc 

Hleh  Total  Pavlod 

'  lac 

Hleh  Total  Pavload 

Qiaraccerlscli 

1:3  2nd 

Multiple  Armament 

2nd 

Multiple  Armament 

Inf luencinx 

3rd 

High  Speed  Capability 

3rd 

Good  Loiter  Capability- 

Selection  of 

1st  Gioice 

4ch 

'Good  Low  Altitude  Performance 

4ch 

IIlRh  Speed  Capability 
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TABLE  VII 


DEFEND  AIRFIELDS 

DEFEND  CITIES 

MISSION  9 

MISSION  10 

1.  1  Squadron  M1C-21D 

1. 

1  Squadron  MIC-21D 

2.  1  Detachment  LiRhtninR 

2. 

1  Squadron  F-5A 

3.  1/2  Detachment  F-4C 

3. 

1/2  Detachment  F-4C 

4.  1  Squadron  F-5A 

4. 

L  Detachment  Li^htninx 

High  Speed  Capability 

All  -tVeaCher  Capability 

1. 

Hlsh  Speed  Capability 
All-Weather  Capability 

3^  Short  Runway  Requirements 

3. 

HiRh  Total  Payload 

4»  Hij^h  Total  Payload 

.  4. 

Multiple  Amainent 

aiOICES  OF  SYSTEMS  OFFERED  FOR  AIR  DEFENSE  MISSIONS 

PAGCAGR  (1  Squadron  -  Approx.  12  aircraft 
1  Decachmenc  •  Approx,  6  aircraft) 


I  SQUADRON  F-5A 

_1  SQUADRON  MIC-21D _ 

11/2  SQUADRONS  HUNTER/ F- 86 

1  DETAa[MENT  LIGHTNING 

1  DETACHMENT  MIRAGE  III 

1  SQUADRON  F-IOOD 

1  SQUADRON  MIG-19 _ 

1/2  DETAQIMENT  F-AC _ 
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capability,  Kemp's  preference  data  implies  that  differences 
in  perfoinnance  characteristics  need  not  equate  to  any 
practical  differences  in  capability.  This  point  is 
substantiated  by  the  clear  preference  for  older,  less 
sophisticated  systems,  such  as  the  A-4F  and  the  B-26K. 

A  final  important  external  variable  affecting  weapons 
capability  is  the  technological  capacity  of  the  owner.  It 
does  little  good  to  have  a  sophisticated  avionics/weapons 
package  in  an  aircraft  if  it  cannot  be  kept  in  operating 
condition.  Such  a  concern  is  minimal  when  assessing  the 
capability  of  weapons  owned  and  operated  by  militarily 
developed  nations  (e.g.,  Israel),  but  for  African  or  Latin 
American  equipment,  technical  capacity  would  certainly 
affect  capability  levels,  most  probably  in  a  negative  way. 
The  chances  of  a  Russian-maintained  MiG-21  having  all  of 
its  systems  in  operating  condition  are  much  better  than  for 
an  Egyptian-maintained  MiG-21  because  the  Egyptians  face 
uncertainties  in  parts  supply,  and  their  maintenance  skills 
are  less  developed.  This  corresponds  to  argiments  presented 
by  Snider  and  Kemp  that  simple,  more  durable  systems  have 

more  military  utility  in  Third  World  environments  than 
45 

sophisticated  ones. 

To  summarize,  defining  capability  using  what  has  been 
termed  the  multi-faceted  approach  involves  integrating  the 
weapon's  performance  characteristics  with  operator  skills, 


^^See  in  particular  Kemp  (19  70)  and  Snider  (1975)  . 
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and  environmental  and  technological  influences.  The 
significance  of  this  integration  is  diagrammed  in  Figure  5 
wherein  capability  is  represented  as  a  vector  in  a  multi¬ 
dimensional  space. 


FIGURE  5 


The  origin,  0,  is  a  theoretical  reference  point  that 
connotes  no  capability  at  all,  while  position  X,Y,Z 
indicates  maximiam  system  capability.  On  this  basis, 
system  #1  has  more  total  capability  than  system  #2  because 
higher  pilot  skills  and  a  more  favorable  technical  environ¬ 
ment  outweigh  system  #2's  built-in  performance  edge.  Such 
an  analysis  is  not  possible  with  the  weapon's  performance 
approach  since  system  #2's  performance  advantage  would 
always  translate  to  higher  capability  relative  to  system  #1. 
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Having  proposed  two  conceptual  definitions  for  aerial 
combat  capability  —  the  first  with  weapons  performance 
characteristics  exclusively,  and  the  second  using  information 
on  the  operator,  environment,  and  performance  capacity  —  it 
becomes  important  to  present  the  strengths  and  weaknesses 
of  each  as  they  pertain  to  measurement  and  scaling. 

Initially  the  weapons  performance  approach  appears  to  offer 
fewer  measurement  problems  because  all  of  the  variables  can 
be  expressed  in  natural  or  derived  units  of  physical  measure, 
e.g.,  length,  width,  etc.  However,  two  significant  uncer¬ 
tainties  must  be  faced  in  the  operationalization  process: 

(1)  determining  if  weighting  the  variables  in  some  fashion 
would  lead  to  a  better  representation  of  capability;  and 

(2)  determining  how  the  variables  should  be  combined  (i.e., 
added,  multiplied,  etc.).  Little  or  no  effort  has  been  made 
to  clarify  these  issues  in  connection  with  capability  assess¬ 
ment.  Hence,  while  using  weapon's  performance  characteristics 
facilitates  measurement  because  they  are  amenable  to  quantifi¬ 
cation,  the  characteristics  themselves  provide  little  guidance 
as  to  how  a  capability  scale  should  be  constructed.^^ 


Solutions  to  both  of  these  questions  are  embodied  in 
the  process  leading  to  factor  scores  where  weighting  is 
determined  by  the  way  a  variable  loads  on  a  factor,  and 
addition  is  rigidly  assumed.  (These  patented  solutions  may 
partially  explain  the  use  of  factor  analysis  by  Muhalka  and 
Snider.)  However,  there  is  nothing  sacred  about  factor 
analysis  or  factor  scores.  The  assumption  that  linearity 
holds  may  be  inappropriate  for  combining  variables  within  a 
factor  to  describe  capability.  Moreover,  if  more  than  one 
factor  is  needed  to  describe  capability,  the  correct  combina¬ 
tion  among  factors  becomes  a  problem.  Questions  can  also  be 
raised  about  the  construct  validity  of  factors .  The  fact  that 
certain  variables  cluster  together  may,  in  actuality,  not 
contribute  to  capability  assessment  at  all. 
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Using  the  weapons  performance  approach  may  also  give 
rise  to  content  validity  problems  to  the  extent  that  key 
variables  may  be  omitted.  To  reinforce  a  point  made 
earlier,  the  analyst  must  cope  with  this  issue  through 
technical  research  on  the  weapon  type.  However,  even  with 
such  an  effort,  there  are  no  assurances  that  meaningful 
information  will  not  be  overlooked. 

Reliability  problems  also  plague  the  performance  approach, 
first  because  of  the  disparities  in  open  source  estimates  of 
the  performance  variables,  and  second  because  of  the  nature 
of  the  variables  themselves .  During  the  course  of  this 
research,  for  instance,  substantive  differences  in  aircraft 
performcince  characteristics  were  commonplace  even  for  the 
most  straightforward  items  such  as  velocity  and  range. 

Without  any  difinitive  sources,  the  chance  for  error  — 
usually  in  the  form  of  an  over-estimate  in  capability  —  is 
high.  Furthermore,  the  dynamic  nature  of  the  variables 
accentuates  the  reliability  problem.  Consider,  for  a 
moment,  the  thrust- to-weight  ratio  (T/W) .  In  some  sources 
it  is  calculated  using  the  normal  take-off  weight  of  the 
aircraft  while  in  others  it  is  derived  using  the  combat 
weight  (i.e.,  the  basic  system  weight  with  one-half  internal 
fuel  and  weapons) .  Since  the  difference  between  these  two 
weight  values  can  be  as  much  as  10-20  per  cent  of  the  total 
system  weight,  T/W  values  vary  tremendously.  In  most  cases, 
the  researcher  has  no  way  of  knowing  which  value  was  used. 

The  same  can  be  said  of  wing  loading,  W/S.  Hence,  the 
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danger  of  misrepresenting  system  capability  using  these 
variables  is  difficult  to  escape. 

With  respect  to  the  multi-faceted  approach,  it  is  felt 
that  using  it  to  scale  capability  can  potentially  yield  a 
more  accurate  representation  of  true  system  capability  and 
better  accommodate  its  dynamic  nature.  However,  the  analyst 
is  again  faced  with  the  problem  of  determining  the  relative 
importance  of  variables .  The  vector  space  illustrated  in 
Figure  5  (p.  69)  assumes  that  weapon  performance  charac¬ 
teristics,  operator  proficiency,  and  the  environment  all 
affect  capability  equally.  In  reality,  this  may  not  be  the 
case. 

One  important  deficiency  found  in  the  multi-faceted 
approach  is  a  genuine  difficulty  in  operationalizing  each 
variable.  While  it  may  be  possible  to  devise  an  algorithm 
for  operator  proficiency  based,  for  example,  on  flight  time 
or  training,  there  are  no  clear  procedures  for  such  attempts 
and  potential  validity  problems .  The  same  can  be  said  for 
operationalizing  technological  capacity  or  the  environment. 
In  the  final  analysis,  these  are  judgemental  variables  with 
no  obvious  physical  correlates  and  as  such  will  always  be 
difficult  to  quantify. 

Given  the  concerns  over  variable  selection/omission, 
weighting,  operationalization,  and  the  like,  using  a  strict 
analytic  procedure  to  derive  capability  scores  may  not  be 
the  best  available  approach.  Judgemental  scaling  techniques 
which  could  tap  the  knowledge  of  weapons  experts ,  systems 
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operators,  etc.,  may  be  more  effective  since  the  subtleties 
of  the  analysis  are  preserved  and  resolved  in  the  judge¬ 
mental  process.  This  proposal  will  be  explored  in  Chapter  IV. 
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IV.  POSSIBLE  METHODS  FOR  SCALING  CAPABILITY 


In  Chapter  II  some  important  reasons  for  developing 
capability  indices  were  mentioned.  Among  the  most  important 
was  that  such  measurement  would  facilitate  side-by-side 
comparisons  with  equipment  from  other  countries  and  provide 
guidance  to  military  and  political  decision  makers  concerning 
the  most  appropriate  weapons  to  transfer  in  order  to  preserve 
or  alter  the  military  balance  in  an  area.  Chapter  III 
presented  two  conceptual  approaches  to  capability  and 
isolated  important  variables  related  specifically  to  aerial 
combat  capability.  Additionally,  the  complexity  of  capability 
measurement  was  stressed  and  the  problems  of  measuring 
capability  discussed.  This  chapter  embodies  the  next  logical 
step  by  presenting  four  scaling  techniques  which  can  be  used 
to  measure  capability  —factor  analysis,  paired  comparisons, 
successive  intervals,  and  multi-attribute  utility  analysis. 
Each  method  is  discussed  separately  with  emphasis  placed  on 
the  rationale  for  using  the  method,  and  its  theoretical 
basis .  Each  method  is  also  applied  to  the  problem  of 
scaling  aerial  combat  capability  and  the  results  of  the 
applications  are  discussed. 

An  important  theme  running  through  this  chapter  is  that 
the  three  judgemental  scaling  techniques  —  paired  comparisons, 
successive  intervals,  and  multi-attribute  utility  scaling  — 
represent  viable  options  to  factor  analysis  for  capability 
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assessment.  In  fact,  it  is  the  author's  contention  that 
they  presently  offer  the  best  solutions  to  measuring 
capability.  There  are  three  fundamental  reasons  for  this 
view.  First,  since  judgemental  techniques  depend  on  expert 
evaluations,  it  can  be  assumed  that  the  resulting  scores 
reflect  the  best  synthesis  of  all  relevant  information. 
Second,  the  analyst  is  not  burdened  with  the  task  of 
selecting  variables  and  specifying  their  exact  inter-  ^ 
relationship.  This  is  done  automatically  in  the  judgemental 
process.  Finally,  using  the  judgemental  technique  allows 
the  analyst  to  consider  more  than  just  performance  charac¬ 
teristics.  Environmental  factors,  tactics,  and  subtle 
variables  such  as  pilot  proficiency,  crew  coordination, 
cockpit  visibility,  and  so  on,  can  be  considered  without 
any  added  burden  to  the  analyst.  The  reader  is  urged  to 
keep  these  advantages  in  mind. 

A.  THE  FACTOR  ANALYTIC  APPROACH 
1.  Rationale 

Despite  periodic  references  to  some  of  the  pitfalls 
encountered  when  trying  to  scale  capability  using  factor 
analysis,  there  are  many  compelling  reasons  for  exploring 
the  technique  in  more  detail. 

First,  as  a  general  scientific  method  for  analyzing 
data,  factor  analysis  has  been  used  with  apparent  confidence 
by  scholars  from  many  disciplines.  Study  and  theorizing  in 
economics,  sociology,  anthropology,  biology,  and  political 
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science,  all  have  been  facilitated  through  a  use  of  factor 
47 

analysis .  The  breadth  of  usage  alone  is  an  enticement  to 
examine  the  approach  fully  before  passing  judgement  on  it. 

Second,  it  is . compatible  with  SPSS  (Statistical 
Package  for  the  Social  Sciences) ,  and  as  such  can  be  used 
easily  and  efficiently  by  the  research  community.  Inasmuch 
as  one  of  the  prevalent  burdens  on  current  research  is 
touching  base  with  vast  amounts  of  data  and  information, 
having  the  option  of  using  an  existing  computer  program  is 
most  attractive. 

A  third,  and  more  pragmatic,  reason  for  exploring 
factor  analysis  is  to  derive  a  set  of  aircraft  capability 
scores  using  the  variables  advanced  in  Chapter  III  in  order 
to  more  fully  examine  the  reasonableness  of  Snider's  and 
Mihalka's  results.  Hopefully,  such  a  comparison  will  be 
heuristic  and  provide  valuable  insight  into  the  strengths 
and  weaknesses  of  the  method. 

2 .  Theory 

As  a  general  methodology,  factor  analysis  can  be 
summarized  with  three  key  concepts 

(1)  the  concept  of  patterned  variation 

(2)  the  concept  of  vector  spaces 

(3)  the  concept  of  dimensionality. 

47 

Rummel,  op.  cit. ,  p.  13. 

"^^Ibid.  ,  pp.  13-19. 
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Discovering  patterns  and  uniformities  in  the 
variation  of  data  is  significant  because  it  is  one  indication 
that  meaningful  relationships  exist.  Factor  analysis 
searches  for  such  variation  in  one  of  two  possible  ways: 
by  scanning  the  cases  of  a  data  matrix  to  see  if  any  exhibit 
similar  characteristics  (Q-analysis) ;  or  by  searching  through 
the  characteristics  themselves  for  regularities  and  patterns 
(R- analysis) .  R- analysis  has  greater  pertinence  to  weapons 
capability  assessment  since  the  only  thing  Q-analysis  would 
accomplish  would  be  a  categorization  of  weapons  by  mission- 
type.  Table  VIII,  which  presents  weapons  performance 
characteristics  for  four  aircraft,  illustrates  this  point. 

TABLE  VIII 


MSPEED 

CEILING 

COMBAT 

RADIUS 

THRUST-TO 

WEIGHT 

WING 

LOADING 

F-15A 

M  2.5 

60  K  ft 

470 

1.25 

55 

F-14A 

2.2 

58  K 

480 

0.86 

40 

A6E 

0.63 

45  K 

800 

0.35 

100 

A7D 

0.67 

45  K 

750 

0.40 

94 

Profile  or  Q-analysis  delineates  two  groups  —  an  air- 
superiority  group  characterized  by  relatively  high  speeds, 
ceilings,  and  thrust- to-weight  ratios,  and  low  wing- loadings 
and  combat  radii;  and  an  attack  grouping  described  by  lower 
speeds,  ceilings  and  thrust- to-weight  ratios,  and  higher 
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combat  radii  and  wing  loading.  These  similarities  are 
apparent  without  factor  analysis.  What  is  less  obvious, 
yet  more  important  for  determining  capability,  is  the 
relationships  among  the  characteristics.  R-analysis 
addresses  this  issue. 

Knowledge  of  vector  algebra  and  vector  spaces  can 
enhance  the  understanding  of  factor  analysis  by  providing 
visual  meaning  to  the  data  and  the  factoring  process. 

The  data  matrix  in  Table  VIII,  for  instance,  can  be  viewed 
as  a  series  of  row  and  column  vectors,  each  having  a  certain 
magnitude  and  spatial  direction.  The  magnitude  depends  on 
the  values  of  the  individual  elements  composing  the  vector, 
e.g.,  MSPEED  =  (2.5,  2.2,  .63,  .67),  while  the  direction 
depends  on  the  particular  relationship  that  variable  has 
with  those  remaining.  Imagine,  for  illustrative  purposes, 
that  the  F-15A,  F-14A,  and  A-6E  can  describe  the  three- 
dimensional  space  given  in  Figure  5.  Each  of  the  five 
characteristics  in  Table  VIII  could  be  plotted  as  vectors 
in  this  space.  The  resulting  angles  between  these  vectors 

would  measure  the  relationships  among  them  for  the  three 

49 

aircraft  describing  the  space.  Small  angular  differences 
approaching  zero  degrees  indicate  a  strong  relationship, 
while  angles  near  90  degrees  suggest  no  correlation  at  all. 


49 

The  basis  for  this  discussion  is  taken  from  Rummel,  R.J., 
"Understanding  Factor  Analysis,"  The  Journal  of  Conflict 
Resolution,  Vol.  XI,  No.  4,  December  1967. 
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In  the  more  extreme  cases ,  obtuse  angles  connote  a  negative 
relationship,  while  a  180  degree  difference  implies  that 
the  vectors  are  inversely  related.  Thus,  in  Figure  5A, speed 
and  service  ceiling  are  strongly  related  while  thrust-to- 
weight  performance  is  much  less  so. 

FIGURE  5 A 


Obviously,  a  geometric  analysis  becomes  impossible 
when  more  than  three  dimensions  are  involved.  Patterns  of 
variation  are  obscured,  clusters  of  vectors  are  not  clearly 
delineated,  and  relationships  are  not  apparent.  The  way 
factor  analysis  deals  with  this  brings  up  the  notion  of 
dimensionality.  As  noted  above,  vectors  which  cluster 
together  are  highly  related.  This  being  so,  it  is  possible 
to  represent  an  entire  cluster  of  variables  with  one 
mathematically  determined  line  —  called  a  dimension  or  factor 
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which  projects  through  the  cluster  and  defines  its  variance. 
In  this  fashion,  many  variables  are  reduced  to  a  manageable 
niimber  of  factors  which  depict  regularities  within  the  data. 
Figure  6  displays  a  plot  of  eight  variables  which  can  be 
represented  with  two  factors,  and  F2 . 


FIGURE  6 


The  angular  separation  between  factors  is  significant 
and  should  be  discussed  briefly.  Notice  in  Figure  6  that 
the  two  factors  describing  the  variable  clusters  are  90  degrees 
apart.  The  fact  that  these  orthogonal  axes  successfully 
represent  the  clusters  implies  that  the  clusters  are 
basically  uncorrelated .  If  the  analyst  suspects  orthogonality, 
or  wishes  to  measure  the  degree  of  orthogonality  in  the  data, 
orthogonal  factoring  (usually  varimax) is  warranted. 

However,  circiimstances  arise  when  orthogonal  factors 
do  not  provide  the  best  representation  of  variable  clusters. 


50 


Rummel , 


op.  cit. , 


1970,  p. 


392  . 
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Figure  7-a,  for  example,  shows  two  variable  clusters  located 

near  each  other  between  orthogonal  axes.  No  matter  how  the 

orthogonal  axes  are  rotated,  they  will  never  precisely 

delineate  the  relationship  between  the  clusters.  The 

general  solution  is  to  drop  the  demand  for  \incorrelated 

factors  and  use  an  oblique  rotation  scheme  (Figure  7-b) . 

However,  as  Rummel  notes,  controversy  exists  over  the  use 

of  oblique  rotation  methods. Some  people  feel  that 

oblique  rotation  is  a  way  to  fabricate  relationships  between 

variables  which,  in  reality,  do  not  exist.  Others,  including 

Rummel,  support  oblique  rotation  on  the  grounds  that  it 

generates  more  precise  information  and  relationships  between 

clusters  and  that  the  process  better  reflects  reality. 

. . . the  real  world  should  not  be  treated  as 
though  phenomena  coagulate  in  unrelated 
clusters.  As  phenomena  can  be  interrelated 
in  clusters,  so  the  clusters  themselves  can 
be  related.  Oblique  rotation  allows  this 
reality  to  be  reflected  in  the  loadings  of 
their  factors  and  their  correlations . 52 

While  there  are  obvious  advantages  gained  through  orthogonal 

factor  rotation  in  terms  of  simplicity  and  amenability  to 

mathematical  manipulation,  they  do  not  justify  dismissing 

oblique  rotation  in  capability  analysis,  especially  since 

capaibility  analysis  is  an  open-ended  issue  at  this  point  in 

time.  Both  approaches  should  be  used. 


51 


■Rummel,  op.  cit.,  1967,  p.  477. 


FIGURE  7-a 
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FIGURE  7-b 
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An  understanding  of  the  three  principles  of  factor 
analysis  —  variability,  vector  representation,  and 
dimensionality  —  provides  the  framework  needed  to  understand 
factor  analytic  scaling  and  measurement.  The  basic 
mechanism  for  factor  analytic  scaling  is  to  calculate 
factor  scores  in  the  following  way.  After  the  patterns  in 
variation  have  been  identified  and  factors  described,  each 
individual  variable  is  weighted  proportionally  to  its 
relationship  with  a  given  factor.  The  greater  or  stronger 
the  relationship,  the  greater  the  weight.  The  value  of 
that  variable  for  a  particular  case  is  then  multiplied  by 
the  weighting  (loading)  and  normalized  to  yield  a  score. 
Since  factors  are  usually  a  composite  of  many  variables, 
i.e.,  a  variable  cluster,  this  weight-times-data  product 
must  be  computed  for  every  variable  associated  with  the 
factor  and  summed  to  provide  the  actual  factor  score. 
Symbolically,  if  a^,  a^,  and  represent  the  loadings  of 
three  variables  on  a  factor,  and  if  the  values  of  these 
variables  are  v^,  V2 /  and  v^  respectively,  the  factor  score, 
S ,  would  be : 


S  =  +  “2^2  “3^3 

In  short,  a  factor  score  measures  the  magnitudes 
of  the  variables  for  a  given  case  and  their  relationships 
to  a  factor.  If  the  factor  has  been  properly  identified, 
and  the  analyst  is  convinced  that  it  legitimately  represents 
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either  in  a  descriptive,  causal,  or  symbolic  sense  —  an 
attribute  or  phenomenon,  the  factor  scores  can  be  used  to 
rate  cases  along  the  factor  with  interval-level  precision. 

3 .  Scaling  Aerial  Combat  Capability 

With  Factor  Analysis 

While  the  theory  behind  factor  analysis  gives 
credence  to  factor  scores,  Snider's  and  Mihalka's  results 
(Figure  3  and  Table  II)  raise  two  questions  that  must  be 
resolved  before  factor  analytic  scores  can  be  accepted  as 
true  representations  of  weapons  capability.  First,  does 
the  two-factor  structure  assumed  by  Snider  and  Mihalka 
sufficiently  represent  a  weapon's  capability,  or  is  this 
structure  a  function  of  the  variables  used?  Second, 
assuming  there  are  multiple  factors,  is  there  any  clear 
way  to  combine  or  weight  them  that  will  lead  to  a  meaningful 
representation  of  the  system's  capability? 

To  answer  the  first  question  a  data  matrix  of 
29  aircraft  (including  both  attack  and  air  superiority 
systems)  and  13  variables  (Snider's  plus  some  of  those 
identified  in  Chapter  III) ,  was  factor  analyzed  using  both 
principal  factor  analysis  and  varimax  rotation.  The 
variables  used  were  defined  in  the  following  way: 

(1)  Maximum  speed  -  the  fastest  advertised  speed  of 
the  system 

(2)  Service  ceiling 
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(3)  Thrust  -  total  maximum  engine  thrust  at  sea- level. 

If  the  system  had  after-burner  capability,  the 
maximum  A/B  thrust  value  was  used. 

(4)  Normal  take-off  weight  -  the  weight  of  the  fuel 
+  platform  +  ordnance  in  its  normal  mission 
configuration . 

(5)  Rate-of-climb  -  measured  in  feet  per  minute  at 
sea  level. 

(6)  Maximum  payload  -  the  maximum  amount  of  ordnance  the 
system  can  carry. 

(7)  Combat  range  -  the  maximum  one-way  distance  the 
aircraft  can  fly  and  perform  its  mission. 

(8)  Combat  Radius  -  the  maximxam  two-way  distance  the 
aircraft  can  fly  and  perform  its  mission. 

(9)  Thrust- to-weight  ratio  -  ratio  of  maximum  engine (s) 
thrust  to  normal  take-off  weight. 

(10)  Wing  loading  -  ratio  of  normal  take-off  weight  to 
total  wing  area. 

(11)  GUNB  -  the  total  number  of  gun  barrels  on  the  aircraft. 

(12)  MISALG  -  a  crude  algorithm  representing  missile 
capability  based  on  the  range  of  the  system  (at 
10-15000  feet  AGL) ,  the  number  of  missiles  the 
aircraft  normally  carries,  and  an  angular  estimate 
of  the  firing  sector. 
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(13)  Production  year  -  the  year  production  began.  If  the 
aircraft  was  an  updated  model,  the  year  the 
modification  occurred  was  used  with  the  thought  of 
capturing  the  incorporated  technological  advances . 

The  resulting  varimax  factor  matrix  and  computed 
factor  scores  are  displayed  in  Tables  IX  and  X.  The  first 
point  to  note  is  the  presence  of  three  distinct  factors  in 
contrast  to  the  two  which  Snider  and  Mihalka  obtained. 

Based  on  the  particular  variables  clustering  together. 

Factor  I  generally  corresponds  to  Snider ' s  Interceptor/ 
Air-to-Air  Combat  -  Factor  (see  Table  IV) .  Similarly,  the 
same  variables  which  load  on  Factor  II  also  load  together 
on  Snider's  Tactical  Support/Ground  Attack  -  Factor.  To 
this  extent,  Snider's  results  have  been  replicated.  However 
all  of  the  variables  not  considered  by  Snider  —  thrust-to- 
weight  ratio,  wing  loading,  GUNB,  and  MISALG  —  cluster 
together  to  form  a  third  factor.  Assuming  that  the  analysis 
in  Chapter  III  is  correct  and  the  added  variables  are 
important  to  air-to-air  combat  capability,  the  presence  of 
the  third  factor  confirms  what  has  previously  been  suggested 
namely,  that  measuring  air-to-air  combat  capability  with 
Factor  I  alone  is  inadequate. 

Some  additional  observations  should  be  mentioned. 
Production  year,  which  loaded  highly  on  the  Air-to-Air 
Combat  -  Factor  for  Snider,  loaded  on  the  new  factor.  This 
is  not  necessarily  inconsistent  since  it  is  argued  that  both 
Factor  I  and  Factor  III  relate  to  the  same  attribute.  Also 
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TABLE  IX 


VARIMAX  ROTATED  FACTOR  MATRIX 
FOR  29  AIRCRAFT  AND  13  VARIABLES 


VARIABLE 

FACTOR  I 

FACTOR  II 

FACTOR  : 

Max  Speed 

.91183 

-.16005 

.15425 

Ceiling 

.90017 

-.14516 

-.10637 

Thrust 

.81375 

.33873 

.27959 

Rate-of-climb 

.85771 

-.17088 

.31725 

Take-off  weight 

.62739 

.68222 

-.04521 

Payload  (max) 

-.22243 

.91291 

.07798 

Combat  range 

-.06186 

.90778 

.01947 

Combat  radius 

-.09686 

.90804 

.00532 

Thrus t-to-weight 

.54453 

-.32122 

.54158 

Wing  loading 

.07857 

.34959 

-.83717 

Number  of 
g\in  barrels 

.07818 

.13349 

.88188 

Missile  algorithm 

.30709 

.24849 

.52984 

Production  Year 

.27103 

.40090 

.52844 
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of  interest  is  the  fact  that  the  thrust- to-weight  ratio 
loeads  equally  on  Factors  I  and  III.  This  seems  to  reinforce 
the  point  just  made  that  both  Factors  I  and  III  are  needed 
to  represent  air-to-air  combat  capability  inasmuch  as  T/W 
is  critical  to  an  air-superiority  platform.  It  also  implies 
that  not  all  variables  will  be  clearly  associated  with  any 
one  factor  in  particular,  at  least  when  employing  varimax 
rotation.  This  represents  a  potential  problem  from  the 
point  of  view  of  defining  and  interpreting  the  factors. 

The  impact  of  the  added  variables  and  the  multiple- 
factor  result  is  more  apparent  in  the  Factor  Score  Matrix 
(Table  X)  and  the  derived  system  rankings  in  Table  XI. 
Depending  on  the  factor  —  or  combination  of  factors  —  chosen, 
the  ranking  of  aircraft  varies.  The  problem  is  obvious.  How 
must  the  factors  be  weighted  and  combined  to  produce  the  best 
results?  With  respect  to  fighter  aircraft,  the  author  tends 
to  favor  the  combination  of  Factors  I  and. Ill  because  the 
variable  clusters  included  are  more  important  to  the  aerial 
combat  mission  than  are  the  variables  in  Factor  II.  However, 
this  cannot  be  considered  a  definitive  equation  for  aerial 
combat  capability  since  the  question  of  weighting  factors 
has  yet  to  be  resolved.  Should  Factor  I  and  Factor  III  be 
viewed  equally,  or  must  Factor  III  be  considered  more 
heavily  than  Factor  I  to  generate  true  capability  scores? 

In  an  attempt  to  focus  more  closely  on  air-to-air 
combat  capability  and  fighter  aircraft,  the  variable  list 
was  reduced  from  thirteen  to  eight,  all  of  which  came  from 
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Table  X 


Factor  Scores  for  29  Aircraft 
13  Variables  Varimax  Rotation 


Aircraft 

Factor  I 

Factor  II 

Factor  ; 

A-37B 

-1.6193 

-  .8519 

-  .2089 

A-4n 

-1.3927 

.2733 

.0683 

A-4E 

-1.5728 

.3169 

.0236 

A-4F 

-1.5728 

.3169 

.0236 

A-6E 

-0.7542 

3.1342 

-1.2066 

A-7D 

-1.0957 

2.0996 

-0.8608 

SU-7B 

-0.3517 

-  .5716 

-  .3454 

SU-7MF 

.1695 

-  .6426 

-  .3271 

Hunter  Mk6 

-1.3381 

-  .6550 

-  .0229 

Lightning  Mk2 

.0471 

-  .8567 

-  .1541 

Lightning  Mk53 

.6861 

.6190 

-  .7096 

Su-9 

.4508 

-  .5456 

-  .6531 

Su-11 

.5492 

-  .8610 

-  .7768 

Faithless 

.6350 

-  .9714 

-  .8727 

Mirage  3C 

.1024 

-  .4651 

.1452 

Mirage  3E 

-  .1252 

-  .3499 

-  .0099 

Mirage  5 

.0938 

-  .0942 

.0117 

Mi(i-19 

-  .5383 

-  .9020 

-  .2701 

MiG-2 IPF 

.0428 

-1.1306 

-  .2305 

MiG-2 IMF 

.3717 

-1.2125 

-  .2252 

MiG- 2 3 

.3456 

-  .2657 

.4424 

MiG- 2 5 

3.0011 

0.5937 

-2.1415 

F-5E 

-  .3523 

-  .3733 

.0570 

F-4B 

.5039 

.7131 

-  .0703 

F-4E 

.7385 

.6729 

-  .0773 

F-14A 

.6848 

.9899 

1.6640 

F-14A  (with 

.7986 

1.3007 

2.6069 

Phoenix) 

F-15A 

1.3223 

.2798 

1.7756 

F-16 

.1699 

-  .5719 

2.3445 
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Table  XI 


Rank  Order  for  12  Fighter  Aircraft 
Using  Different  Factor  Combinations 


Factor  I  alone 


Factor  I  +  Factor  III 


MiG-25 

F-15A 

F-4E 

F-14A  (with  Phoenix) 

F-14A 

MiG- 2 3 

MiG- 2 IMF 

F-16 

Mirage  3C 
MiG-21PF 
F-5E 
MiG- 19 


F-15A 

F-14A  (with  Phoenix) 

F-16 

F-14A 

MiG- 2 5 

MiG-23 

F-4E 

Mirage  3C 
MiG- 2 IMF ^ 

MiG-21PF 
F-5E 
MiG- 19 


All  Factors 


F-15A 

F-14A  (with  Phoenix) 

F-14A 

F-16 

MiG- 2 5 

F-4E 

MiG- 2 3 

Mirage  3C 

F-5E 

MiG- 2 IMF 
MiG-2 IPF 
MiG- 19 
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Chapter  III.  Additionally,  the  number  of  aircraft  was 
reduced  to  provide  a  more  homogeneous  set  of  21  interceptor/ 
air  superiority  platforms.  These  changes  also  serve  to  test 
the  three-factor  structure  and  the  various  rank-orderings 
obtained  previously. 

The  factor  matrices  for  the  varimax  and  oblique 
rotations  of  the  eight-variable  data  set  are  presented 
in  Tables  XII  and  XIII.  While  the  basic  variable  patterns 
are  similar  to  those  obtained  in  the  first  factor  analysis, 
a  number  of  changes  have  occurred.  First,  the  thrust-to- 
weight  ratio  now  loads  strongly  on  only  one  factor  instead 
of  moderately  on  two.  Conversely,  wing  loading  relates 
moderately  to  two  factors  in  the  latest  analysis  rather 
than  one.^^  Shifts  like  this  can  be  expected  with  changes 
in  cases  and  variables,  and  their  effect  on  capability 
assessment  and  the  factor  scores  is  noticeable.  Table  XIV 
shows  the  rank-ordering  obtained  from  the  second  set  of 
varimax  factor  scores.  Comparing  these  rankings  with  the 

previous  rankings  in  Table  XI  for  corresponding  factor 
54 

combinations  shows  that  occasionally  a  system  will  shift 


^All  29  aircraft  were  factor  analyzed  in  terms  of  the 
eight  variable  as  well.  In  this  instance  thrust-to-weight 
ratio  loaded  equally  on  two  factors  (Factors  I  and  II)  rather 
than  on  one  factor.  On  the  other  hand,  GUNB  loaded  on 
Factor  II  alone  instead  of  the  dual-loading  exhibited  in 
Table  XII.  This  shows  that  the  variable  clusters  depend  on 
the  types  of  aircraft  analyzed.  The  complete  results  are 
contained  in  Appendix  I . 

54 

For  comparisons  with  Table  II,  consider  Factor  I  on 
Table  XI  and  Table  XII  to  be  the  same,  and  Factor  III  of 
Table  XI  as  equivalent  to  Factor  II  on  Table  XII. 
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Table  XII 


Varimax  Rotated  Factor  Matrix 
For  21  Interceptor/Air  Superiority  Aircraft 


Variable  Factor  I 

Max  Speed  .8952 

Ceiling  .9103 

Combat  Radius  .3237 

Thrust- to- weight  .1068 

Wing  Loading  .3256 

GUNB  -.1836 

MISALG  .0335 

Production  Year  .0944 


Factor  II 

Factor  III 

.1564 

.1961 

-.1347 

-.0115 

.1509 

.6607 

.8973 

-.2058 

-.5500 

-.4816 

.7031 

.5484 

.0422 

.8435 

.7286 

.3567 
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Table  XIII 


(Oblique  Rotation) 

Factor  Pattern  Matrix  For 
21  Interceptor/Air  Superiority  Aircraft 


Variable 

Factor  I 

Factor  II 

Factor  III 

Max  Speed 

.1875 

.8863 

-.1754 

Ceiling 

-.0716 

.9140 

.0043 

Combat  Radius 

.0532 

.2947 

-.6652 

Thrus  t-to-weight 

.9696 

.1032 

.3270 

Wing  Loading 

-.4515 

.3535 

.4357 

GUNB 

.6065 

-.2161 

-.4846 

MISALG 

-.1138 

-.0019 

-.8717 

Production  Year 

.6887 

.0699 

-.2792 

Factor  Correlations 

Factor  I 

Factor  II 

Factor  III 

Factor 

I 

1.0000 

-.0497 

-.2945 

Factor 

II 

-.0497 

1.0000 

-.0321 

Factor 

III 

-.2945 

-.0321 

1.0000 
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Table  XIV 


Rank  Order  for  12  Fighter  Aircraft 
Using  Different  Factor  Combinations 


Factor  I  alone 

MiG-25 

F-15A 

MiG- 2  IMF 

F-14A 

MiG-23 

Mirage  3C 

F-4E 

F-14A  (with  Phoenix) 

MiG-2lPF 

F-16 

F-5E 

MiG- 19 


Factors  I  and  II 

F-15A 
F-16 
MiG-25 
F-14A 
MiG- 2 3 
MiG- 2 IMF 

F-14A  (with  Phoenix) 

MiG-21PF 

F-4E 

Mirage  3C 
F-5E 
MiG- 19 


Factor  II  alone 

F-16 

F-15A 

F-14A 

MiG-23 

F-14A  (with  Phoenix) 

MiG- 2 IMF 

F-5E 

MiG-2 IPF 
F-4E 

Mirage  3C 

MiG-25 

MiG-19 


All  Factors 

F-14A  (with  Phoenix) 

F-15A 

MiG- 2 5 

F-14A 

F-16 

MiG-23 

Mirage  3C 

F-4E 

MiG- 2 IMF 
MiG- 2 IPF 
F-5E 
MiG- 19 


Note:  For  Comparisons  with  Table  XI,  Factor  I  on  this  Table 

corresponds  with  Factor  I  on  Table  XI  while  Factor  II 
corresponds  to  Factor  III  on  Table  XI. 
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as  much  as  four  positions  in  either  direction.  The  oblique 
factor  scores  display  even  greater  disparity.  (See 
Appendix  I . ) 

Just  how  significant  these  differences  really  are 
is  an  intriguing  question.  In  many  instances,  fairly  high 
rank  correlation  (as  measured  by  Spearman's  rho)  exists 
betwen  two  corresponding  rankings.  On  the  other  hand, 
the  magnitudes  of  the  differences  are  not  as  important  as 
the  fact  that  no  compelling  reasons  can  be  offered  for  using 
one  factor  method  over  another. 

In  contrast  to  the  easily  interpreted  clusters 
obtained  when  thirteen  variables  are  used  (Table  IX) , 
interpreting  the  variable  clusters  from  the  eight-variable 
analysis  is  more  challenging.  Factor  III  (Table  XII) ,  for 
example,  belies  exact  interpretation  since  it  unites  two 
important  yet  conceptually  different  variables  —  combat 
radius  and  missile  capacity.  Another  source  of  confusion 
is  whether  to  treat  Factors  I  and  II  as  representing  a 
weapon  type  (e.g.  interceptor)  or  a  weapon  attribute  (e.g. 
energy  or  maneuverability) .  Snider  and  Mihalka  both  use 
the  factors  to  identify  aircraft  types.  However,  with  three 
factors  appearing,  even  when  just  one  type  of  aircraft  is 
considered,  the  attribute  interpretation  gains  credence. 

At  any  rate,  these  uncertainties  constitute  additional 
stumbling  blocks  in  capability  analysis. 
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The  preceding  investigation  of  the  factor  analytic 
method  suggests  three  things: 

(1)  When  analyzing  aircraft,  the  number  of  factors  obtained 
depends  on  the  variables  selected.  Thus,  the  assump¬ 
tion  that  two  clear  dimensions  will  completely 
describe  a  weapon's  capability  cannot  be  supported 

as  a  general  condition. 

(2)  Occasionally,  important  variables  will  not  load  on 
any  one  factor  clearly.  This  creates  factor- 
interpretation  problems  and  raises  questions  as  to 
the  validity  of  factor  scores. 

(3)  The  ranking/scoring  of  aircraft  depends  on  the  factor 
combinations  used.  Until  something  is  resolved  as  to 
the  proper  weighting  and  combination  of  multiple- 
factor  situations,  factor  score  results  must  be  viewed 
with  caution. 

B.  THE  METHOD  OF  PAIRED  COMPARISONS 
1.  Rationale 

A  fair  amount  of  work  has  been  done  by  psychologists 
to  develop  methods  which  measure  qualitative  phenomena  such 
as  beauty,  affects,  and  excellence,  all  of  which  are  difficult 
to  define  because  they  have  no  simple  physical  correlates. 

In  a  very  real  sense,  scaling  capability  presents  the  same 
problem  as  scaling  attitudes.  It  was  noted,  for  instance, 
that  selecting  variables  and  specifying  their  exact  relation¬ 
ships  was  difficult  in  the  weapons  performance  approach. 
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The  same  was  true  of  the  multi-faceted  approach  which  had 
additional  operationalization  problems.  Inasmuch  as  judge¬ 
mental  techniques  depend  on  the  expertise  of  the  judges, 
the  operationalization  problems  are  eased  since  it  is  assiomed 
that  the  appropriate  information  will  be  weighted  and 
processed  automatically  in  the  judgemental  process.  Judge-  — 
mental  techniques  may  be  particularly  valuable  to  policy 
analysts  who  either  do  not  have  access  to  technical  data, 
or  for  one  reason  or  another,  do  not  feel  qualified  to  select 
variables  by  themselves. 

Of  the  many  judgemental  scaling  techniques  available, 
the  paired  comparisons  method  is  particularly  advantageous 
for  several  reasons.  First,  it  is  simple  to  administer. 
Information  can  be  gathered  either  by  questionnaire  or 
through  direct,  oral  responses.  Second,  in  comparison  to 
other  judgemental  techniques,  such  as  the  constant  sum  method 
and  the  subjective  estimate  method,  it  asks  little  of  the 
judges.  All  that  is  assumed  is  that  a  judge  can  rank  a  pair  ^ 
of  cases  according  to  which  has  the  greater  or  lesser  amount 
of  a  particular  attribute.  This  equates  to  ordinal-level 
information.  Finally,  the  method  has  a  certain  amount  of 
elegance  to  the  extent  that  the  requested  ordinal  information 
is  transformed  to  interval  measurement  by  making  some  simple 
assumptions  about  a  judge's  behavior.  Capability  scores 
would  thus  have  the  same  precision  as  a  factor  score  and 
potentially  be  more  accurate. 
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2 .  Theory 


Central  to  understanding  the  paired  comparisons 
method  of  scaling  is  the  law  of  comparative  judgement,  a 
set  of  statistical  equations  derived  by  psychologist  Louis 
L.  Thurstone.  Thurstone  postulated  that  when  a  stimulus 
or  instance  is  presented  to  an  observer  it  gives  rise  to  a 
"discriminal  process"  which  has  a  value  to  the  observer  on 

55 

a  psychological  continuiim.  Because  of  fluctuations  in 
the  observer's  perception/ judgement,  the  same  instance  will 
not  result  in  precisely  the  same  value  all  the  time.  However 
there  is  a  definite  regularity  in  the  estimating  process  such 
that  the  frequency  distribution  of  judged  values  for  a  given 
instance  over  a  large  number  of  trials  has  a  normal 
distribution.^^  A  graphic  example  of  the  theoretical 
distributions  of  discriminal  processes  for  two  instances, 

_i  and  2,  is  shown  in  Figure  8.  The  mean  of  each  distribution 
and  S j ,  is  the  scaled  value  of  each  instance,  while  the 
standard  deviations,  cr^  and  a ^ ,  determine  their  variance. 

Since  an  individual's  judgement  of  an  instance  or 
attribute  has  a  normal  distribution  for  a  large  number  of 
trials,  it  follows  that  the  difference  between  two  instances 
presented  simultaneously  would  also  have  a  normal  distribu¬ 
tion.  It  turns  out  that  the  mean  of  such  a  distribution 


^^Torgerson,  Warren  S.,  Theory  and  Methods  of  Scaling, 
p.  159,  John  Wiley  &  Sons,  1967. 

^^Ibid.,  p.  159. 
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FIGURE  8 


equals  the  difference  between  the  means  of  the  original 
distributions,  -  S ^ .  Its  standard  deviation,  a^,  depends 

in  turn  on  the  standard  deviations  of  the  two  original 

distributions  (a^  and  a.),  and  on  the  correlation  between 

/  2  2 

them,  r...  Mathematically,  a ,  =  /a .  +a.  -2r..a.a. 

2.J  a  2.  J  2.2  2.  3 

If  instances  _i  and  are  judged  a  large  number  of 
times,  the  proportion  of  times  one  is  rated  higher  than  the 
other  can  be  ascertained.  For  all  practical  purposes,  this 
proportion  can  be  viewed  as  the  probability  that  the  high¬ 
rated  instance  will  continue  to  be  rated  superior  in  futiare 
trials.  Treating  the  proportional  results  as  probabilities 
is  convenient  because  it  permits  the  analyst  to  enter 
standard  normal  tables  and  convert  the  information  to 
z-scores.  In  other  words,  if  in  Figure  8  instance  2  is 
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judged  superior  to  instance  the  difference  between  the 

scale  values,  and  S^,  can  be  measured  with  a  z-score 

value,  ,  which  relates  to  the  proportion  of  times  instance 

2  was  judged  superior  to  instance  2* 

Since-  there  is  a  certain  amount  of  variance  in  the 

difference  between  Sj  and  —  measured  in  terms  of  the 

standard  deviation,  a,  —  the  z. .  values  will  exhibit  the 
d  2.3 

same  variance.  Hence,  the  difference  between  and  is 
more  precisely  written  as  Substituting  the  complete 

expression  for  a^,  the  equation  for  the  law  of  comparative 
judgment  results. 


S  .  -  S.  =  z.  .  -  2  r^.o.a.  (10) 

where  S^  and  S^  are  psychological  scale  values  of  the  com¬ 
pared  instances  i  and  2/  is  the  standard  normal  deviate 

associated  with  the  number  of  times  instance  2  was  judged 
greater  than  instance  i,  is  the  standard  deviation 
of  stimulus  i,  is  the  standard  deviation  of  instance  2, 
and  r^j  is  the  correlation  between  the  deviations  of  the  two 
instances . 

As  presented  above,  the  equation  describing  the 
law  of  comparative  judgment  is  not  solvable  since  the  num¬ 
ber  of  unknowns  exceeds  the  number  of  possible  equations  that 
can  be  generated.  Hence,  certain  simplifying  assumptions 
are  normally  made  to  arrive  at  a  workable  set  of  equations. 
Usually  it  is  assumed  that  the  standard  deviation  for  the 
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This  reduces 


discriminal  differences,  a,,  is  constant. 

d 

equation  (10)  to 


S  .  —  S .  =  cz  .  . 
3  1  ID 


which,  by  allowing  the  constant,  c,  to  equal  1,  becomes. 


^i  "  ^ij 


(11) 


Although  a  simplified  version  of  the  law  of  comparative 
judgment,  equation  (11)  can  be  used  with  confidence  to  derive 
scale  values  for  the  instances  or  objects  that  are  judged. 

Thus,  if  there  are  n  instances,  n-1  equations  of  the  same 
form  as  equation  (11)  will  be  obtained  which,  when  solved 
simultaneously,  give  interval-level  scores  for  every  instance. 

3 .  Scaling  Aerial  Combat  Capability  Using  Paired  Comparisons 

To  test  the  applicability  of  the  paired  comparisons 

58 

method  to  capability  analysis,  a  questionnaire  was  distri¬ 
buted  to  34  Navy  and  Air  Force  officers  at  the  Naval  Post¬ 
graduate  School  asking  them  to  rate  nine  fighter  aircraft 
according  to  the  aircraft's  aerial  combat  capability.  The 
nine  aircraft  —  MiG-19,  MiG-21,  MiG-23,  Mirage-3C,  F-4E, 

F5E,  F-14A  (without  Phoenix) ,  F-15A,  and  F-16  —  were  chosen 


^"^Ibid.  ,  p.  165. 

^^A  copy  of  the  questionnaire  is  provided  in  Appendix  II. 
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because  of  their  current  and/or  potential  use  in  Third 
World  areas,  and  most  especially  in  the  Middle  East.  Care 
was  taken  to  insure  that  every  judge  had  a  basic  familiarity 
with  every  system  by  providing  performance  data  on  each  air¬ 
craft.  However,  it  was  stressed  that  the  judges  were  to 
rely  on  their  personal  knowledge  and  experience  and  not  to 
base  their  decisions  solely  on  the  provided  data. 

To  establish  a  reference  scenario  for  the  judges, 
it  was  stipulated  that  the  combat  environment  was  limited 
to  altitudes  below  20  thousand  feet  and  speeds  between 
MO. 5-1. 5.  These  particular  limits  were  based  on  open  source 
documentation  that  the  majority  (i.e.,  85-90  percent)  of 

all  aerial  combat  has  occurred  within  these  altitude  and 
59 

speed  regimes .  In  order  to  accentuate  aerial  combat 
capability,  it  was  also  stipulated  that  no  long-range 
missiles  such  as  Phoenix  were  available.  This  was  also  done 
to  more  accurately  represent  current  Third  World  capability. 
Finally,  all  pilots  were  assumed  to  be  of  equal  ability. 

Since  the  most  compelling  reason  for  using  a  judg¬ 
mental  scaling  technique  in  the  desire  to  tap  expert  opinion 
and  experience,  the  selection  of  judges  is  constrained  by 
the  particular  system  under  consideration.  For  this  reason 
33  of  the  34  officers  who  participated  in  the  survey  were 
aviators  with  fighter  experience.  The  one  exception  was  a 


59 


Aviation  Week  and  Space  Technology,  p.  41,  12  July  1971. 
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Navy  pilot  who,  although  not  trained  as  a  fighter  pilot, 
possessed  a  unique  knowledge  of  several  of  the  Soviet  sys¬ 
tems  gained  through  direct  conversations  with  pilots  who 
had  defected  from  the  Soviet  Union. 

The  raw  and  transformed  data  gathered  in  the  survey 
are  presented  in  Tables  XV,  XVI  and  XVII.  In  Table  XV,  the 
raw  frequency  matrix,  each  individual  block  entry  reflects 
the  number  of  times  the  aircraft  at  the  top  of  a  particular 
column  was  judged  superior  to  an  aircraft  at  the  beginning 
of  a  particular  row.  Thus,  reading  down  the  F-16-column, 
the  22  indicates  that  22  of  the  34  judges  felt  the  F-16 
was  superior  to  the  F-15A  in  terms  of  aerial  combat  capa¬ 
bility.  The  corresponding  blocks  in  Tables  XVI  and  XVII 
represent  the  same  information  as  a  proportion  and  a  z-score 
respectively . 

Once  the  z-score  array  is  calculated  scale  values 
are  derived  by  taking  column  averages  of  the  z-score  values. 
Note,  however,  that  the  z-score  matrix  has  a  number  of  missing 
values.  This  occurs  whenever  one  aircraft  within  a  given 
pairing  is  judged  superior  to  the  other  by  every  judge  making 
the  corresponding  z-scores  t<”.  Despite  this  problem,  scale 
values  can  still  be  calculated  by  finding  the  average  z-score 
difference  between  all  pairs  of  z-scores  in  adjacent  columns. 
More  precisely,  if  j  and  k  are  adjacent  columns,  and  9^^^ 


^^This  procedure  is  emphasized  because  it  is  felt  that 
this  problem  will  be  a  common  one  in  weapons  assessment. 
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Table  XV 


Raw  Frequency  Matrix 

Paired  Comparisons 
of  Nine  Fighter  Aircraft 


MiG 


F-16 

F15A 

F14A 

21 

F-16 

- 

12 

1 

0 

F-15A 

22 

- 

15 

1 

F-14A 

33 

19 

- 

0 

MiG 

21 

34 

33 

34 

- 

MiG 

23 

32 

32 

33 

17 

F-4E 

34 

34 

34 

21 

F5E 

34 

34 

34 

21 

Mir 

3C 

34 

34 

32 

20 

MiG 

19 

34 

34 

34 

25 

MiG  Mir-  MiG 


23 

F4E 

F5E 

3C 

19 

2 

0 

0 

0 

0 

2 

0 

0 

0 

0 

1 

0 

0 

2 

0 

17 

13 

13 

14 

9 

- 

12 

13 

17 

12 

22 

- 

9 

13 

14 

21 

25 

- 

24 

17 

17 

21 

10 

- 

9 

22 

20 

17 

25 

_ 
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Table  XVI 


Proportional  Matrix 

Paired  Comparisons 
of  Nine  Fighter  Aircraft 


F-16 

F-15A 

F-14A 

MiG 

21 

MiG 

23 

F-4E 

F-5E 

Mir 

3C 

MiG 

19 

F16 

- 

.353 

.03 

0 

.059 

0 

0 

0 

0 

F-15A 

.647 

- 

.441 

.03 

.059 

0 

0 

0 

0 

F-14A 

.970 

.559 

- 

0 

.03 

0 

0 

.059 

0 

MiG 

21 

1.0 

.970 

1 

- 

.5 

.382 

.382 

.412 

.265 

MiG 

23 

.941 

.941 

.970 

.5 

- 

.353 

.382 

.5 

.353 

F-4E 

1 

1 

1 

.618 

.647 

- 

.265 

.382 

.412 

F-5E 

1 

1 

1 

.618 

.618 

.735 

- 

.706 

.5 

Mir 

3C 

1 

1 

.941 

.588 

.5 

.618 

.294 

- 

.265 

MiG 

19 

1 

1 

1 

.735 

.647 

.588 

.5 

.735 

- 
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Table  XVII 


Z-Value  Matrix 

Paired  Comparisons 
of  Nine  Fighter  Aircraft 


F-16 

F-15A 

F-14A 

MiG 

21 

MiG 

23  F-4E 

F-5E 

Mir 

3C 

MiG 

19 

F-16 

- 

-.377 

-1.88 

- 

-1.563  - 

- 

- 

- 

F-15A 

.377 

- 

-  .148 

-1.88 

-1.563  - 

- 

- 

- 

F-14A 

1.88 

.148 

- 

- 

-1.88  - 

- 

-1.352 

MiG 

21 

- 

1.88 

- 

- 

0  -.3 

-.3 

-.222 

-.628 

MiG 

23 

1.563 

1.563 

1.88 

0 

-  -.377 

-.3 

0 

-.377 

F-4E 

- 

- 

- 

.3 

.377  - 

-.628 

-.3 

-.222 

F-5E 

- 

- 

- 

.3 

.3  .628 

- 

.542 

0 

Mir 

3C 

- 

- 

1.352 

.222 

0  .3 

-.542 

- 

-.628 

MiG 

19 

- 

- 

- 

.628 

.377  .222 

0 

.628 

- 
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is  the  set  of  rows  having  entries  in  both  column  j  and 

column  k,  then  the  average  scale  difference  S, -S .  is 

^  3 


Z 

iee  . 


= 


(z  -  z  .  .  ) 
ik  13 


(12) 


With  this  format/  eight  equations  with  nine  unknowns 
result. However,  since  the  scale  produced  is  interval, 
the  origin  can  be  set  arbitrarily.  In  this  instance  assigning 
a  value  to  any  one  of  the  aircraft  accomplishes  the  same 
thing.  Thus,  using  the  derived  equations  (Appendix  II),  and 
arbitrarily  giving  the  MiG-19  a  score  of  1.0,  leads  to  the 
following  ranking  and  scale  values : 


Aircraft 

Score 

F-16 

4.67 

F-15A 

3.80 

F-14A 

3.21 

MiG- 21 

1.63 

Mirage  3C 

1.46 

MiG- 2 3 

1.44 

F-4E 

1.27 

F-5E 

1.02 

MiG-19 

1.00 

In  terms  of  the  rankings  obtained  with  factor  analysis,  this 
arrangement  correlates  most  closely  with  the  ranking  based 
solely  on  Factor  II  in  Table  XIV.  (Spearman's  rho  =  .97.) 


^ ^Appendix  II 


contains  the  entire  list  of  equations. 
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A  review  of  Table  XII  shows  that  thrust- to- weight  ratio ,  wing 
loading,  GUNB,  and  production  year,  all  load  highly  on  this 
particular  factor.  On  this  basis,  it  is  reasonable  to  con¬ 
clude  that  some,  if  not  all,  of  these  particular  factors 
were  weighted  more  heavily  than  others  in  the  judgmental 
process . 

Because  of  the  small  sample  size,  no  claim  can  be 
made  that  these  particular  paired  comparison  results  actually 
reflect  the  preferences  of  the  Navy  and  Air  Force  Fighter 
communities.  It  is  possible  that  changes  in  the  ranking 
would  occur  if  more  judges  were  polled.  However,  based  on 
the  large  interval  between  the  F-16/15A/14A  and  the  other 
systems,  it  is  safe  to  suggest  that  none  of  them  would  be 
ranked  below  any  of  the  remaining  systems  and  further  that 
any  changes  that  did  occur  would  be  among  the  lowest  six. 

In  sum,  while  it  is  important  to  realize  that  the 
level  of  measurement  is  not  improved,  paired  comparisons 
does  represent  a  viable  option  to  factor  analysis. 

C.  THE  METHOD  OF  SUCCESSIVE  INTERVALS 
1.  Rationale 

The  method  of  successive  intervals  is  another  judg¬ 
mental  scaling  technique  that  may  have  a  place  in  capability 
analysis.  Aside  from  the  general  advantage  of  relying  on 
knowledgeable  judges  to  analyze  capability,  which  it  shares 
with  the  paired  comparisons  method,  the  most  immediate  reason 
for  demonstrating  the  method  at  this  time  is  to  obtain  a  set 
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of  results  to  compare  with  those  obtained  with  paired  com¬ 
parisons.  Consistent  results,  while  not  necessarily  verifying 
either  judgmental  technique,  will  at  least  provide  some 
indication  of  their  reliability. 

The  method  is  also  worth  considering  because  it  pro¬ 
vides  information  in  a  format  which  may  be  more  useful  than 
paired  comparison  results  in  certain  instances.  Specifically, 
it  scales  category  boundaries,  e.g.,  fair,  good,  excellent, 
on  the  same  interval  scale  as  the  objects  themselves.  Thus, 
the  analyst  obtains  information  on  the  judges'  estimates  of 
the  meaning  of  a  particular  score,  and  not  just  the  score 
itself. 

2 .  Theory 

Like  the  method  of  paired  comparisons,  the  method  of 
successive  intervals  relies  on  several  theoretical  assumptions 
about  the  judges'  behavior.  These  assumptions  are: 

(1)  that  the  psychological  continu\im  a  judge 
uses  to  assess  the  values  of  instances  can 
be  divided  into  a  series  of  successively 
ordered  categories 

(2)  that  due  to  natural  fluctuations  in  the 
judge's  perception  the  category  boundaries 
are  not  located  in  a  fixed  point  but,  rather, 
project  a  normal  distribution  of  positions 

on  the  continuum 

(3)  that  the  judge  will  place  a  stimulus  or 
instance  below  any  category  boundary  whenever 
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the  value  of  the  stimulus/instance  on 
the  continuum  is  less  than  the  category 

j  62 

boundary . 

Taken  together,  these  assumptions  mean  that  the 
boundaries  between  categories  behave  like  stimuli.  A 
graphic  interpretation  of  the  theoretical  distributions  for 
an  instance/object,  and  two  boundaries,  and  B^,  is 

foiind  in  Figure  9. 

FIGURE  9 


As  noted  in  the  previous  discussion  on  comparative  judgement, 
the  difference  between  two  normally  distributed  values  will 
also  project  a  normal  distribution.  Consequently,  if  B^ 
represents  the  mean  value  of  the  upper  boundary,  and  is 
the  mean  value  of  the  instance,  Bg  "  will  be  the  mean 
value  of  the  distance  between  the  boundary  and  instance, 

— 5 - 2 - 

and  a,  =  /(j.  +  a  -  2r.  a.cr  ,  its  standard  deviation, 

d  1  g  ig  1  g 

^^Ibid. ,  p.  206. 


no 


Notice  that  this  is  the  same  expression  encountered  in  the 
derivation  of  the  law  of  comparative  judgement  with  boundary 
information  substituted  for  the  second  instance.  After  many 
judgements  of  the  distance  between  and  a  z-score  can 
be  obtained  reflecting  the  proportion  of  times  the  instance 
was  placed  below  the  category  boundary.  Following  the  same 
steps  which  led  to  the  law  of  comparative  judgement  provides 
the  expression  for  the  law  of  categorical  judgement  as  well: 


-  S  .  =  z  .  /a  , 

g  1  ig  : 


-  2r.  a 
ig 


(13) 


Again,  simplifying  assumptions  must  be  made  to  obtain 
solutions.  In  most  cases,  it  is  not  unreasonable  to  assume 
that  the  instance-value  and  the  boundary  are  stochastically 
independent  random  variables  with  a  correlation  coefficient, 

r.  =0.  Moreover,  it  can  also  be  generally  assumed  that 

2 

all  bounds  have  the  same  variance,  so . that  =  c.  (Other 

assumptions  can  be  made  depending  on  the  circumstances  and 
data.  It  is  important  to  remember  that  they  dictate  the 
data  analysis  procedures  required  to  obtain  scale  values). 
Applying  the  previous  assumptions,  the  simplified  form  of 
the  law  of  categorical  judgement  becomes: 


(14) 


^^Ibid. ,  p.  209. 


Ill 


Given  n  instances  and  m  category  upper  boundaries /  mn  equa¬ 


tions  in  the  same  form  as  equation  (14)  can  be  derived.  Be¬ 
cause  no  assumptions  are  made  about  the  variance  of  solv¬ 

ing  this  set  of  equations  is  more  difficult  than  is  the 
paired  comparisons  procedure.  The  interested  reader  is 
referred  to  Appendix  III  for  the  recommended  solution. 

3 .  Scaling  Aerial  Combat  Capability  Using 

Successive  Intervals 

The  same  sample  of  Navy  and  Air  Force  pilots  used  in 
the  paired  comparisons  scaling  process  participated  in  the 
successive  interval  scaling  experiment  as  well.  The  aircraft 
scaled,  combat  parameters,  and  environment  also  were  held 
constant.  However,  instead  of  pairing  one  aircraft  against 
another,  the  judges  were  asked  to  categori ze  the  aircraft 
as  being  either  excellent,  above  average,  average,  below 
average,  or  poor  with  respect  to  its  aerial  combat  capability. 
Appendix  III  contains  a  copy  of  the  questionnaire  used. 

The  raw  frequency  matrix  is  presented  in  Table  XVIII. 
With  the  exception  of  one  aircraft  (the  MiG-19) ,  the  judgments 
of  each  system  clustered  within  two  to  three  categories. 

The  fact  that  the  judgmental  distributions  are  not  spread 
over  a  wider  range  of  categories  indicates  a  certain  level 
of  agreement  among  the  judges.  However,  it  makes  the  deri¬ 
vation  of  capability  scores  more  troublesome.  In  fact, 
interval  scores  cannot  be  calculated  for  the  F-14A,  F-15A, 
and  F-16  because  the  method  requires  a  frequency  spread  of 
at  least  three  categories.  This  is  a  distinct  weakness  in 
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Table  XVIII 


MiG-19 

MiG- 21 

MiG- 2 3 

F-4E 

F-5E 

Mirage 

F-14A 

F-15A 

F-16 


Poor 

1 

0 

0 

0 

0 

0 

0 

0 

0 


Raw  Frequency  Matrix  For 
Successive  Interval  Judgments 

Below  Ave.  Average  Above  Ave. 


14 

15 

3 

3 

15 

16 

9 

11 

14 

9 

16 

9 

6 

18 

15 

3 

16 

15 

0 

0 

18 

0 

0 

9 

0 

0 

1 

Excellent 

1 

0 

0 

0 

0 

0 

16 

25 

33 
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the  method.  (Notice,  however,  that  the  F-14A,  F-15A,  and 
F-16  can  be  rank-ordered  according  to  their  distributions.) 


One  further  concession  is  required.  Since  a  raw 


frequency  of  zero  leads  to  a  z-score  it  is  necessary  to 
compress  the  five  original  categories  down  to  three  (below 
average,  average,  and  above  average)  in  view  of  the  fact 
that  all  but  one  of  the  remaining  six  aircraft  exhibit  this 
distribution  in  the  first  and  last  columns.  It  is  important 
to  stress  that  this  reduction  does  not  affect  the  score  of 
any  aircraft  except  the  MiG-19,  and  that  to  an  insignificant 
degree.  Eliminating  categories  is  simply  an  adjustment  to 
the  fact  that  "poor"  and  "excellent"  are  not  pertinent  to  the 
aircraft  listed. 


After  eliminating  the  F-14A,  15A,  and  16,  and  reducing 


the  number  of  columns,  the  frequency  matrix  appears  in  Table 
XIX.  Converting  the  raw  frequencies  to  z-scores  and  using 
the  mathematical  procedure  outlined  in  Appendix  III,  leads 
to  the  following  rank  order  and  scores : 


Aircraft 


Score 

No  interval  score  possible. 
Ranking  is  based  on 
original 

raw  frequency  data 


F-16 

F-15A 

F-14A 


MiG- 19 


MiG- 21 
Mirage  3C 
MiG- 2 3 


F-5E 

F-4E 


.401 

.338 

.124 

-.023 

-.188 

-.694 
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Table  XIX 


Adjusted  Raw  Frequency  Matrix 
For  Successive  Interval 
Technique 


Below  Above 


Ave 

Average 

Ave 

MiG-19 

15 

15 

4 

MiG-21 

3 

15 

16 

MiG- 2 3 

9 

11 

14 

F-4E 

9 

16 

9 

F-5E 

6 

18 

10 

Mirage  3C 

3 

16 

15 
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Notice  that  the  only  difference  between  this  ranking  and 
the  one  obtained  with  paired  comparisons  is  the  reversal 
between  the  F-4E  and  F-5E.  Using  Spearman's  rho  again, 
this  equates  to  a  rank  difference  correlation  of  0.983. 

Using  the  derived  categoiry  boundary  values  of  -.839 
and  .467  (see  Appendix  III)  it  is  interesting  to  note  that 
all  of  the  six  systems  scaled  turn  out  to  be  "average". 

This  information  gives  meaning  to  the  scale  values  and 
suggests  that  the  differences  between  the  aircraft  are  slight. 
Figure  10  diagrams  the  final  result. 

To  s-ummarize,  the  successive  interval  method  gives 
the  analyst  more  information  than  either  the  paired  compari¬ 
son  method  or  factor  analysis  without  a  reduction  in  preci¬ 
sion.  However,  it  cannot  be  used  effectively  to  scale  a 
group  of  weapons  when  there  are  clearly  superior  systems 
within  the  group  because  the  dispersion  of  judgments  will  be 
too  narrow  to  determine  category  bounds .  In  this  sense  it 
is  not  as  robust  as  the  previous  methods.  Nevertheless, 
in  situations  where  the  preferences  among  weapons  are  less 
obvious,  it  has  great  utility  and  helps  the  analyst  interpret 
the  scale  values  more  readily. 

D.  THE  MULTI-ATTRIBUTE  UTILITY  APPROACH 
1 .  Rationale 

While  the  factor  analytic,  paired  comparisons,  and 
successive  interval  methods  all  provide  interval-level  capa¬ 
bility  measurement,  it  is  often  desirable  to  have  even  more 
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precision.  The  most  obvious  need  occurs  when  the  analyst 
wishes  to  expand  from  a  side-by-side  comparison  of  weapons 
capability  to  a  comparison  of  the  military  capability  of 
one  country  with  another.  Recall,  for  example,  the  problems 
with  Mihalka's  country  capability  scores  because  of  their 
derivation  from  interval  data.  Another  useful  calculation 
that  cannot  be  performed  with  interval  scores  is  percentage. 
This  becomes  important  when  the  analyst  needs  to  know  how 
much  better  one  system  is  than  another. 

It  is  clear  that  things  like  coiintry  capability 
scores  or  percentage  comparisons  are  desired,  the  researcher 
must  strive  for  ratio  measurement.  None  of  the  preceeding 
techniques  can  accommodate  this  demand.  However,  one  possi- 
ble  avenue  of  approach  that  could  produce  a  "conditional" 
ratio  scale  is  multi-attribute  utility  theory  (MAUT) .  It 
should  be  stressed  that  certain  assumptions  must  be  made  to 
generate  ratio  information,  some  of  which  can  be  challenged. 

In  the  author's  view,  however,  the  assumptions  are  not  un¬ 
reasonable.  Thus,  as  a  potential  approach  to  the  ratio-data 
question.  Multi-attribute  utility  theory  warrants  some 
attention. 

A  second  important  reason  for  investigating  MAUT  is 
that  it  accommodates  an  analysis  of  the  multi-faceted  defini¬ 
tion  of  capability  better  than  any  of  the  previous  techniques. 
Thus,  even  if  its  use  as  a  ratio-measurement  device  can  be 
challenged,  it  still  merits  consideration  as  an  analytic 
technique. 
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2 .  Theory 


Utility  or  value  theory  is  a  set  of  axioms  designed 
to  facilitate  the  decision  process.  Basically,  it  depends 
on  the  assumption  that  the  decisionmaker  will  act  rationally 
and  always  choose  a  course  of  action  which  maximizes  expected 
utility  (or  usefulness)  as  defined  by  the  decisionmaker's 
goals  and  the  environmental  constraints  imposed  upon  him. 

When  applied  to  the  decision  process,  utility  theory  requires 
that  all  possible  decision  outcomes  be  quantified,  the  utility 
of  each  defined,  and  a  decision  reached  based  on  maximizing 
utility. 

Central  to  the  entire  process  is  the  derivation  of 
a  utility  function,  u,  which  assigns  a  real  value  to  each 
possible  consequence  such  that  the  utility  of  consequence  b, 
u(b),  is  greater  than  the  utility  of  consequence  c,  u(c) , 
if  and  only  if  consequence  b  is  preferred  to  consequence  c.^^ 
It  is  important  to  realize  that  the  utility  function  depends 
on  the  subjective  judgment  of  the  decisionmaker  and  of  his 
perception  of  the  environment  and  the  decision  objectives. 
Equally  important  is  the  realization  that,  once  defined,  the 
utility  function  acts  as  an  evaluative  scale  by  which  all 
possible  outcomes  are  measured. 

Until  recently,  most  of  the  precepts  in  utility 
theory  related  to  decisions  which  were  based  on  a  single 


^^Keeney,  R.L.,  Multidimensional  Utility  Functions; 
Theory,  Assessment  and  Application,  p.  16,  MIT,  1969. 
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attribute  or  criterion  such  as  profit.  As  Keeney  and  others 
have  pointed  out,  however,  few  decisions  are  based  on  just 
one  measure  of  effectiveness.^^  This  realization  has 
prompted  the  development  of  multi-attributed  utility  theory 
and  other  procedures  which  attempt  to  cope  with  more  complex 
problems.  In  view  of  the  multidimensional  nature  of  capa¬ 
bility,  this  particular  facet  of  general  utility  theory 
deserves  further  explanation. 

According  to  Winterfeldt  and  Fischer,  "multi-attribute 
utility  theory  (MAUT)  combines  a  class  of  psychological 
measurement  models  and  scaling  procedures  that  can  be  applied 
to  the  evaluation  of  alternatives  with  multiple  value  rele¬ 
vant  attributes For  example,  MAUT  can  be  used  to  analyze 
preferences  between  cars  described  by  the  attributes  cost, 
comfort,  prestige,  and  performance.  Similarly,  it  could  be 
used  to  analyze  weapons  systems  according  to  the  series  of 
specified  attributes  which  define  their  capability. 

A  possible  model  for  the  employment  of  MAUT  in 
weapon's  capability  assessment  is  suggested  by  the  design 
engineering  process  advocated  by  design  engineers  to  optimize 
system  design  and  maximize  system  worth.  As  outlined  by 
Kline  and  Lifson,^^  this  process  basically  involves: 


^^Ibid. ,  p.  9. 

^^Winterfeldt,  D.  and  Fischer,  G.W. ,  Multi-Attribute  Utility 
Theory:  Models  and  Assessment  Procedures,  p.  1,  NTIS,  1973. 

^"^English,  John  M.  ,  (ed.),  Cost-Effectiveness,  The  Economic 

Evaluation  of  Engineered  Systems,  pp.  79-80,  John  Wiley  & 

Sons,  1968. 
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(1)  Obtaining  a  clear  statement  of  the  goal 

or  purpose  of  the  system  and  the  environment 
in  which  it  is  to  operate 

(2)  selecting  performance  criteria  which  best 
define  the  objectives  and  assigning  measures 
of  utility  to  them  to  describe  how  valuable 
each  criterion  is 

(3)  comparing  and  weighting  the  various  criteria 
to  put  their  utility  functions  on  a  common 
basis  or  scale.  (Once  this  is  done,  the 
individual  utility  functions  can  be  combined 
into  one  objective  function  which  can  be  used 
to  calculate  total  system  worth.) 

(4)  Using  the  output  from  steps  1-3/  various 
alternatives  are  examined  in  light  of  esti¬ 
mated  states  of  nature  and  an  optimal  solu¬ 
tion  is  chosen  based  on  trying  to  maximize 
expected  utility. 

While  the  purpose  of  the  design  engineer's  analysis 
is  finding  an  optimal  system  to  fulfill  an  objective,  the 
solutions  obtained  can  also  be  used  to  evaluate  or  scale 
existing  systems.  In  other  words,  the  optimal  solution  can 
be  considered  as  an  ideal  model  against  which  all  other 
systems  can  be  judged. 

This  usage  can  be  illustrated  using  fighter  aircraft. 
Step  1  calls  for  a  clear  statement  of  the  system's  purpose 
and  the  operating  environment.  For  illustrative  purposes  —  and 
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to  be  consistent  with  previous  scaling  done  in  the  thesis  — 
suppose  the  stated  purpose  is  air-to-air  combat.  Addition¬ 
ally,  consider  the  operational/technical  environment  to  be 
similar  to  that  found  in  the  Middle  East  with  the  system 
designed  to  maximally  perform  at  altitudes  below  twenty  thou¬ 
sand  feet  AGL  and  at  speeds  between  M  0.5  -  1.5.  (The 
reasonableness  of  these  parameters  has  been  addressed  pre¬ 
viously.)  These  specifications  direct  and,  in  effect,  constrain 
the  judges  to  consider  only  what  is  important  for  maximizing 
air-to-air  combat  capability  under  these  conditions.  Thus, 
the  analyst  possessed  a  way  to  account  for  significant  re¬ 
gional  factors,  tactics,  or  any  other  pertinent  variables. 

After  clarifying  the  environment  and  purpose  of  the 
system.  Step  2  calls  for  selecting  criteria.  As  with  Step  1, 
this  can  be  done  in  general  terms  or  in  great  detail;  but 
whatever  level  is  chosen,  guidance  should  be  obtained  from 
experts  (e.g.,  aeronautical  engineers,  pilots,  etc.)  or 
through  a  detailed  technical  analysis.  The  analysis  performed 
in  Chapter  III  provides  the  list  of  criteria  used  here. 


Platform  Criteria  Weapon  Criteria 

Max  speed  (energy)  Range 


Acceleration  (T/W) 

Maneuverability  (W/S) 

Endurance  (combat 
radius) 


Missile  speed 
Firing  envelope 

No.  of  guns 


Miscellaneous  Criteria 


Technological  level 
of  the  country 

Pilot  proficiency 
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At  this  point  the  analyst  must  obtain  from  a  series 
of  subjective  evaluations  by  his  judge (s),  a  utility  func¬ 
tion  defining  the  utility  of  each  criterion  over  a  range  of 
values.  For  the  capability  problem  and  criteria  tabulated 
above,  the  analysis  must  be  done  at  two  levels: 

(1)  Level  I .  Determine  the  individual  utility 
function  for  each  of  the  criteria  under  the 
three  main  dimensions,  platform,  weapon ,  and 
miscellaneous .  For  this  particular  problem 

this  will  lead  to  ten  separate  utility  functions. 

(2)  Level  II.  Determine  the  utility  relationships 
among  the  three  dimensions  as  they  relate  to 
air-to-air  combat  capability. 

The  bi-level  procedure  thus  leads  to  an  over-all  expression 
relating  the  utility  of  each  criterion  to  the  ultimate 
objective,  air-to-air  combat. 

The  actual  process  used  to  derive  a  utility  function 
merits  some  attention.  Consider,  for  illustrative  purposes, 
maxim\im  speed  and  define  it  as  attribute  x.  The  utility 
function,  u(x) ,  can  be  obtained  over  a  range  of  values  in 
the  following  way: 

(1)  First,  determine  the  upper  and  lower  limits 
(x*  and  X*)  of  the  attribute,  x.  The  lower 
limit,  X*,  will  be  based  on  what  the  judge 
feels  is  the  minim\am  speed  required  for  an 
air-superiority  aircraft.  The  upper  limit. 
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X*,  will  be  based  on  what  the  judge  feels  is 
technologically  possible/desirable.  By  defini¬ 
tion,  the  lower  limit  marks  the  threshhold 
where  the  utility,  u(x)  =  0,  while  the  upper 
limit  has  a  utility,  u(x)  =1.0. 

(2)  Next,  determine  the  general  shape  of  the  uni¬ 
dimensional  utility  function  by  defining  the 
tradeoffs  within  each  criterion.  This  is  done 
by  presenting  the  judge  with  a  series  of  choices 
(lotteries)  involving  the  criterion  to  be 
measured.  (In  the  jargon  of  utility  theorists, 
this  is  the  same  as  finding  out  if  there  is 
"risk  aversion,"  "increasing/decreasing  risk 
aversion,"  etc.  These  topics  are  fully 
covered  by  Keeney.)^® 

(3)  Finally,  quantitatively  assess  the  relative 
utilities  of  several  speed  values.  This  will 
result  in  a  hypothetical  curve  such  as : 


^^Keeney,  R.L. ,  Multidimensional  Utility  Functions;  Theory, 
Assessment  and  Application,  pp.  19-22,  MIT,  October,  1969. 


since  the  curve  represents  an  expert's  judgment 
of  the  utility  of  various  speed  capabilities, 
it  can  be  used  to  scale  the  speed  capability  of 
existing  systems.  Thus,  if  an  aircraft  had  a 
maximum  speed  of  Ml. 5,  it  would  have  a  utility 
of  0.75,  based  on  the  judge's  preferences. 


Once  the  utility  functions  for  each  of  the  criteria 
are  derived,  the  important  questions  of  how  best  to  combine 
and  weight  each  of  the  fxinctions  must  be  addressed.  Put 
another  way,  the  trade-offs  among  the  criteria  must  be  dis¬ 
cerned.  Weighting  requires  a  judge's  estimate  of  the  relative 
importance  of  each  criterion.  The  combination/simplification 
process  is  completely  determined  by  the  presence  or  absence 
of  three  properties  within  the  utility  functions,  utility 
independence,  pairwise  preferential  independence,  and  pariwise 
marginality . These  concepts  can  be  defined  as  follows 
(1)  Utility  independence.  Assume  consequence 


utility  independent  of  xj  if  the  decision¬ 
maker's  relative  preferences  for  x^  with  xj 
held  fixed  are  the  same  regardless  of  the 
chosen  value  of  x-:-. 


^^Giaque,  William  C.,  Prevention  and  Treatment  of  Strepto¬ 
coccal  Sore  Throat  and  Rheumatic  Fever  —  A  Decision  Theoretic 
Approach,  Ph.D.  Thesis,  p.  IV-18,  Harvard  University,  Nov.  1^72 . 

“^^Ibid.  ,  pp.  IV-15-19. 
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(2) 


Pairwise  preferential  independence  is  exhibited 
if  the  choice  between  two  consequences 


does  not  depend  on  the  values  of  x-,  x  , 

J  n 

for  all  pairs  of  attributes. 


(3)  Pairwise  marginality  holds  if  lottery  (choice) 

(x^,Xj) , (x^ ,x^)  is  indifferent  to  (x^,x^ ) , (x^ , x^ ) , 
where  lottery  A,B  is  a  choice  situation  with 
the  probabilities  of  consequences  A  and  B  both 


one-half. 


Multiplication  is  called  for  if  there  is  both  utility  indepen¬ 
dence  and  pairwise  preferential  independence.  Under  these 


conditions,  the  combined  utility  function,  u(xi  becomes, 


n 


1  +  ku(x)=  n  [1  +  kk.u. (X. ) ] 


(15) 


where  u(x)is  a  multi-attributed  utility  function  (e.g.  platform), 
k  and  k^^  are  constants  with  k  >  -1  and  0  <  k^  <  1,  Uj^(x^)  is 
the  utility  function  of  an  individual  criterion  (e.g.,  speed), 
and  IT  is  the  symbol  for  multiplication.  Addition  holds  when 
utility  independence,  pairwise  preferential  independence, 

and  pairwise  marginality  are  all  present.  In  this  case  the 

—  72 

combined  utility  function,  u(x)is. 


^^Ibid. ,  p.  IV-19. 


72. 


Ibid. 
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(16) 


__  n 

u(x)  =  Z  k.u. (x. ) 
i=l  ^  ^  ^ 

where  u(x)  is  again  a  multi-attributed  utility  function 
(e.g./  platform^  weapon/  etc.),  is  a  constant,  and  u^(x^) 
is  the  unidimensional  utility  fiinction  of  a  particular  cri¬ 
terion  (e.g./  speed/  thrust-to-weight  ratio,  etc.). 

As  with  the  individual  criteria,  the  three  struc¬ 
tural  attributes,  platform,  weapon,  and  miscellaneous ,  are 
also  tested  for  utility  independence,  pairwise  preferential 
independence,  and  pairwise  marginality  to  determine  their 
combinatorial  relationship.  The  resulting  equation  will 
describe  air-to-air  combat  in  terms  of  utility  values  for 
each  of  the  criteria  which  define  it  and  will  either  be  in 
the  form  of  equation  (15)  or  (16) .  At  this  stage,  however, 
Ui(Xi)  is  now  a  multi-dimensional  function  corresponding  to 
either  platform,  weapon ,  or  miscellaneous  criteria  elements, 
instead  of  just  a  unidimensional  function  for  a  single 
criterion . 

It  is  possible  that  the  requirements  for  addition  or 
multiplication  will  not  be  present,  or  that  they  will  not  be 
decipherable.  If  such  is  the  case  most  sources  recommend 

uding  the  additive  form  since  it  is  generally  a  good  approxi- 

73 

mation  of  the  multi-attribute  utility  function. 


73 


Keeney, 


op.  cit. , 


pp.  23-25. 
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One  of  the  stated  reasons  for  exploring  MAUT  was 
the  chance  that  it  could  provide  an  acceptable  ratio  scale 
for  capability.  Assximptions  are  necessary  because  utility 
scaling  is  normally  associated  with  interval  measurement. 

In  most  applications  this  is  an  inescapable  condition  since 
the  origin  and  unit  are  selected  arbitrarily.  Recall,  how¬ 
ever,  that  in  the  derivation  of  utility  curves  for  capability 
assessment  a  zero-point  is  demanded  from  the  judge  and  de-  ^ 
fined  as  the  threshold  value  at  which  the  attribute  ceases 
to  be  useful  to  the  specified  objective.  At  least  as  far  as 
the  judge  is  concerned,  therefore,  this  is  an  absolute  zero- 
point.  If  the  analyst  is  willing  to  generalize  the  validity 
of  this  point  it  can  be  considered  a  "natural"  origin  and 
lead  to  ratio  measurement. 

A  second  major  assumption  that  must  be  made  to  allow 
ratio  measurement  is  that  the  judge  is  capable  of  making 

reasonably  precise  judgments  of  the  value  of  an  attribute. 

74 

Lifson  argues  that  utility  measurement  is  not  exact. 

People  make  mistakes  and  judgments  are  sometimes  inconsistent. 
However,  as  demonstrated  previously,  judgmental  measurement 
theory  postulates  a  definite  regularity  in  value  judgments. 

On  this  basis,  it  can  be  argued  that  enough  precision  exists 
to  measure  at  ratio  level.  (Some  judgmental  measuring  tech¬ 
niques,  e.g.,  the  s\ibjective  estimate  and  constant  sum  methods. 


^^Lifson,  Melvin  W.,  Application  of  Criteria  and  Measures 
of  Value  in  Engineering  Design,  Ph.D.  Thesis,  University  of 
California,  Los  Angeles,  p.  85,  1965. 
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Nevertheless , 


actually  produce  ratio  scales,  for  example.) 
the  analyst  must  recognize  the  possible  hazards  involved 
in  the  assumptions  as  well  as  the  fact  that  utility  theory 
does  not  formally  claim  ratio  measurement  is  possible. 

3 .  Scaling  Aerial  Combat  Capability  with  Multi- 

Attribute  Utility  Functions 

To  demonstrate  the  MAUT  approach,  utility  curves 
were  obtained  for  the  ten  criteria  set  forth  previously. 

Two  judges  provided  the  necessary  information.  The  first 
was  an  experienced  fighter  pilot  with  over  1000  hours  in 
flight  time  and  a  graduate  of  the  Navy  Fighter  Weapons  School. 
His  preferences  are  depicted  in  Figures  11-19.  The  second 
judge  was  a  student  at  the  Naval  Postgraduate  School  with  a 
degree  in  National  Security  Affairs.  His  judgments  are 
represented  in  Figure  20,  which  scales  the  technological 
capacity  of  various  countries. 

Platform  utility  values  were  calculated  for  the  nine 
aircraft  rated  in  previous  sections  by  evaluating  their  per¬ 
formance  parameters  against  the  utility  functions  in 
Figures  11-14.  Weights  (the  k-values)  were  assinged  to  each 
of  the  criteria  by  the  judge,  and  since  none  of  the  three 
simplification  properties  could  be  identified  clearly,  an 
additive  solution  was  used.  The  values  derived  were:^^ 


^^Torgerson,  op .  cit. ,  pp.  61-117. 

^^All  calculations  are  included  in  Appendix  IV. 
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FIGURE  11 


u(x^) 


UTILITY  CURVE  OF  SPEED 


FIGURE  12 

UTILITY  CURVE  OF  T/W 


130 


FIGURE  13 


UTILITY  CURVE  FOR  WING  LOADING 


FIGURE  14 

UTILITY  CURVE  FOR  COMBAT  RADIUS 
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FIGURE  15 


UTILITY  CURVE  FOR  MISSILE  RANGE 


FIGURE  16 

UTILITY  CURVE  FOR  MISSILE  SPEED 
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FIGURE  17 


UTILITY  CURVE  FOR  MISSILE  FIRING  ENVELOPE 


FIGURE  18 

UTILITY  CURVE  FOR  NO.  OF  GUN  BARRELS 
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FIGURE  19 


UTILITY  CURVE  FOR  PILOT  PROFICIENCY 


FIGURE  20 

UTILITY  CURVE  FOR  COUNTRY  TECHNOLOGICAL  CAPACITY 

i.or  _ — 


.75- 


.5 


.25- 


0  - ^ - , - 1 - 1 - ^ ^ 

Egypt  Israel  Japan  Bin. tain  USSR  U.S. 

Korea  India  France 

Sweden 
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Aircraft 


Platform  Utility  u  (x  ) 

- - - P-^P^ 


F-16 

F-15 

F-14 

F-4E 

F-5E 


.96 

.86 

.79 

.47 

.48 

.69 

.68 

.62 

.58 


Mirage  3C 
MiG- 19 
MiG- 21 
MiG-23 


Once  platform  utility  has  been  determined,  the  calcu¬ 


lation  is  keyed  to  a  particular  country  or  situation  by  the 
nature  of  the  remaining  criteria.  For  example,  the  weapons 
utility  score  (derived  from  Figures  15-18)  will  depend  on 
the  country,  tactics  and  situation  under  study.  The  same  is 
true  for  the  miscellaneous  utility  value  since  it  embodies 
pilot  proficiency  and  the  technological  capacity  of  a  co\intry. 


To  continue  the  calculation,  some  hypothetical  data 


for  Israel  and  Egypt  were  used  to  demonstrate  the  effects 
on  system  capability  that  accrue  from  a  consideration  of 
weapons  and  miscellaneous  factors.  The  data  assumed  are: 


(1)  Two  types  of  aircraft  are  present,  the  MiG-21 
and  F-4E 

(2)  Israeli  pilots  have  an  average  of  1000  hours 
in  flight  time. 

(3)  Egyptian  pilots  have  an  average  of  500  hours 
in  flight  time. 
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(4)  All  aircraft  missile  and  gun  systems  are 
comparable . 

Using  the  weights  obtained  from  the  judges  (Appendix  IV) , 
and  determining  a  multiplicative  relationship  to  hold  between 
platform  utility,  weapon  utility,  and  miscellaneous  utility, 
result  in  the  F-4E  having  a  utility  value  of  .52  and  the 
MiG-21  of  .40.  When  compared  to  the  original  assessments 
of  their  platform  utilities,  these  values  are  strikingly 
different.  This  suggests  that  while  the  MiG-21  may  physically 
be  a  better  aircraft  than  the  F-4E  (in  terms  of  aerial  combat) , 
Israel's  edge  over  Egypt  in  pilot  proficiency  and  technical 
ability  actually  make  the  Israeli  F-4E  a  superior  system. 

Obviously,  such  subtlety  is  not  possible  with  any 
other  technique  discussed  in  this  thesis.  If  reliable  utility 
functions  can  be  obtained,  MAUT  offers  the  greatest  opportun¬ 
ity  for  realistic  capability  assessment  since  it  takes  into 
account  more  than  just  the  weapons. 

If  the  analyst  can  accept  the  assumptions  needed 
for  ratio  measurement,  these  scores  can  be  multiplied  by 
inventory  levels  to  obtain  a  measure  of  the  usefulness  of  a 
country's  fighter  force  in  performing  air-to-air  combat. 

Simple  arms  transfer  calculations  can  also  be  performed  as 
well.  For  example,  returning  to  the  utility  values  just 
derived  for  the  F-4E  and  MiG-21,  if  Israel  had  50  F-4E's 
and  Egypt  55  MiG-21 's,  the  total  air-to-air  combat  capability 
for  each  country  —  measured  in  utils  —  would  be: 
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Israel:  (.52)  (50)  =  26.0 
Egypt;  (.4)  (55)  =  22.0 

Continuing,  if  the  U.S.S.R.  sent  Egypt  ten  MiG-21 's,  the 
relative  capabilities  of  the  two  countries  (again  measured 
in  utils)  would  be  the  same.  A  supplier  such  as  the  U.S. 
would  then  have  to  send  at  least  eight  F-4E's  to  Israel  to 
reinstate  her  previous  capability  advantage.  It  should  be 
stressed  that  until  the  reliability  of  MAUT  can  actually  be 
tested,  it  is  not  appropriate  to  advocate  its  wholesale 
adoption  for  measuring  and  comparing  capabilities  in  this 
fashion.  At  this  point,  however,  MAUT  shows  greater  promise 
for  ratio  comparisons  than  any  of  the  other  techniques 
reviewed  in  this  thesis. 
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V.  CONCLUSIONS 


The  major  concern  driving  this  research  has  been  the 
desire  to  improve  the  measurement  and  assessment  of  arms 
transfers.  Several  approaches  are  available,  but  one  of  the 
most  valuable,  and  the  one  given  most  attention  in  this 
thesis,  is  capability  assessment.  Developing  ways  to  measure 
capability  has  merit  not  only  for  political  research,  but  for 
military  intelligence  estimates  as  well,  and  this  broad 
usage  gives  impetus  to  the  task. 

In  the  course  of  this  research,  several  important  con¬ 
clusions  have  been  reached.  First,  the  current  use  of  factor 
analytic  techniques  to  generate  capability  scores  for  weapons  — 
and  in  particular,  aircraft  —  should  be  reassessed.  Not 
only  has  there  been  a  tendency  to  oversimplify  and  misre¬ 
present  aircraft  capability,  but  there  is  a  tendency  to 
misuse  the  scores  once  they  have  been  derived.  As  interval- 
level  data,  factor  scores  can  only  be  used  for  side-by-side 
comparisons  of  similar  systems  and  cannot  be  used  to  calculate 
composite  or  country  capability  scores.  Too  much  blind  faith 
has  been  placed  in  the  method  because  of  its  frequent  us  in 
data  analysis  problems  and  impressive  theoretical  framework, 
but  not  enough  emphasis  has  been  placed  on  the  crucial  tasks 
of  selecting  variables,  interpreting  factor  results,  and 
weighting  and  combining  multiple-factor  solutions.  As  this 
thesis  has  shown,  without  such  guidance  any  one  of  a  number 
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of  scores  can  be  derived  for  a  weapon  which  can  change 
its  comparative  ranking  with  similar  systems . 

Reassessment  does  not  mean  abandonment.  If  decisions 
could  be  made,  or  at  least  a  consensus  reached,  on  some  of 
these  issues,  factor  analysis  would  become  an  extremely 
valuable  tool  for  arms  transfer  measurement.  Through  SPSS 
it  can  bring  the  benefits  of  computer  processing  to  bear 
on  important  research  efforts. 

Given  the  problems  encountered  in  factor  analysis,  a 
second  important  conclusion  reached  in  this  thesis  is  that 
judgmental  scaling  techniques,  in  particular  the  paired  com¬ 
parisons  and  successive  interval  methods,  are  a  viable  alter¬ 
native  to  factor  analysis  for  capability  assessment.  In 
this  research  a  high  rank  order  correlation  (rho  =  .983)  was 
obtained  when  these  techniques  were  used  to  scale  aerial 
combat  capability  in  nine  modern  fighter  aircraft.  More 
importantly,  they  eliminate  the  major  problems  encountered 
in  factor  analysis  without  sacrificing  precision. 

Of  the  two,  it  was  discovered  that  the  successive  inter¬ 
val  method  was  sensitive  to  certain  distribution  patterns 
in  the  judges'  responses.  If  judgmental  variation  is  less 
than  three  categories  (intervals),  bo\andaries  and  scores 
cannot  be  calculated.  This  says  that  interval  scores  cannot 
be  determined  for  any  system  which  is  universally  agreed 
upon  as  superior  or  inferior  to  another  system.  Although 
this  restricts  the  application  of  the  successive  interval 
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method  to  situations  involving  very  similar  systems,  it  is 
an  extremely  powerful  method  in  that  the  analyst  gains 
information  on  the  meaning  of  the  scores  along  with  the 
scores  themselves. 

One  glaring  weakness  this  thesis  uncovers  in  previous 
empirical  studies  which  use  capability  scores  is  the  ten¬ 
dency  to  use  interval  data  as  ratio-data.  Many  researchers 
would  like  to  be  able  to  measure  capability  on  a  ratio  scale 
because  it  would  allow  absolute  comparisons  of  the  military 
potentials  among  countries.  This  thesis  addresses,  but  does 
not  adequately  solve,  measuring  capability  at  a  ratio-level. 
Multi-attribute  utility  theory  provides  the  best  avenue  to 
a  ratio  scale,  but  certain  assumptions  must  be  made  which 
reduce  the  authoritativeness  of  the  results. 

Ratio  measurement  aside,  multi-attribute  utility  scaling 
is  an  impressive  analytical  approach  which  transforms  human 
judgment  into  mathematical  assessment.  With  respect  to  arms 
transfers,  it  is  the  only  technique  comprehensive  enough  to 
deal  with  capability  as  more  than  just  a  combination  of 
performance  characteristics.  Hence,  it  should  be  given 
more  than  a  cursory  glance  by  military  and  political  analysts. 

If  this  work  attests  to  anything,  it  is  the  fact  that 
human  judgment,  and  not  a  computer,  is  the  key  to  capability 
assessment.  The  challenge  to  future  analysis  will  be  to  make 
such  judgments  more  precise  by  reaching  agreement  on  what  capa¬ 
bility  actually  means.  The  two  conceptual  appraoches  suggested 
in  this  thesis  are  a  start  in  this  direction. 
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APPENDIX  I 


Factor  Analysis  Results 


Table  A 

Varimax  Rotated  Factor  Matrix 
For  29  Aircraft  and  8  Variables 


Variable 

Max  Speed 

Ceiling 

Combat  Radius 

Thrust-to-weight 

Wing  loading 

Niomber  of 
gun  barrels 

Missile  Algorithm 

Production  Year 


Factor  I 
.9486 
.94610 
-.25510 
1 .5923 
-.03127 

-.0160 

.2292 

.2112 


Factor  II 
.1412 
-.1008 
-.2630 
.5762 
-.8903 

.8511 

.4231 

.3093 


Factor  III 
.0962 
.0038 

I  -81371 

-.0319 

.0711 

.2860 
.4999 
I  .7180| 
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Table  B 


Oblique  Rotated  Factor  Matrix 
For  29  Aircraft  and  8  Variables 


Variable  Factor  I  Factor  II 


Max  Speed 

.0099 

.  9576 

Ceiling 

-.2343 

.9835 

Coinbat  Radius 

-.2853 

■.2133 

Production  Year 

.2477 

.1917 

Thrust- to-weight 

.5160 

1 

.53511 

Wing  loading 

-.9199 

.0807 

Gun  Barrels 

.8646 

.1175 

Missile  Algorithm 

.3759 

.1918 

Factor 

Correlations 

Factor 

I  Factor  II 

Factor  I 

1.0000 

.2542 

Factor  II 

.2542 

1.0000 

Factor  III 

.1446 

.0393 

Factor  III 
.  0537 
-.0178 
.8502 
.  6883 
-.1021 
.1513 
.2130 
.4582 


Factor  III 
.1446 
.0393 
1.0000 
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Table  C 


Rank  Order  for  12  Fighter  Aircraft 
Using  Didder ent  Factor  Combinations 
(8  Variables  —  Oblique  Rotation) 


Factor  I  alone 
F-16 

F-14A  (with  Phoenix 

F-15A 

F-14A 

MiG- 2 3 

Mirage  3C 

Mig-21  MF 

MiG-2lPF 

F-4E 

MiG-19 

F-5E 

MiG- 2 5 


Factor  I  +  Factor  II 

F-15 

F-16 

F-14A  (with  Phoenix) 

F-14A 

MiG- 2 IMF 

MiG- 2 5 

MiG-23 

Mirage  3C 

MiG-2lPF 

F-4E 

F-5E 

MiG-19 


Factor  11+  alone 

MiG-25 

F-15A 

F-14A  (with  Phoenix) 
F-16 

MiG- 2 IMF 
MiG-2 IPF 
F-14A 
MiG-23 
Mirage  3C 
F-4E 
MiG-19 
F-5E 


All  Factors 

F-14A  (with  Phoenix) 

F-15A 

F-16 

F-14A 

MiG- 2 3 

MiG-25 

F-4E 

Mirage  3C 
MiG- 2 IMF 
F-5E 

MiG- 2 IPF 
MiG- 19 
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APPENDIX  II 


CALCULATIONS  AND  EQUATIONS  FOR  THE  PAIRED  COMPARISON  METHOD 


(1.) 

(2.) 

(3.) 

(4.) 

(5.) 

(6.) 

(7.) 

(8.) 


Equations  for  Scale  Values 
(Paired  Comparisons) 


^F16  "  ^F15 
^F15  “  ^F14  ^ 

^F14  "  ^M21 
^M21  "  ^M23 
^M23  “  ^MIR-3C 
^MIR-3C  "  ^F4E 
^F4E  “  ^F5E 
^F5E  "  ^M19 


.593 


3.084  + 

1.658 

3 

.43  +  .509 

5 

-.826 

+  .704 

5 

.948 

-  .173 

4 

-.155  + 

1.142 

4 

-1.77  + 

1.855 

4 


^  1.58 

188 

=  -.0244 

=  .194 

.247 

.021 
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FIGHTER  CAPABILITY  MEASUREMENT 


The  pijrpose  of  this  questionnaire  is  to  gather  data  on 
fighter  aircraft  capability  for  a  project  on  arms  transfers 
currently  under  study  by  JCS  and  members  of  the  Government 
Department  at  NPS .  For  each  of  the  36  pairs  of  aircraft 
listed  below  circle  the  one  you  consider  to  be  superior  in 
terms  of  aerial  combat  capability.  Consider  each  pair  separ¬ 
ately.  Assume  a  combat  environment  between  5-20K  ft  and 
speeds  between  M  0 . 5-1. 5 .  Also  assume  that  no  long-range 
missiles  such  as  Phoenix  are  available  and  that  pilots  are 
approximately  of  equal  ability. 

Some  performance  data  has  been  provided  to  insure  a 
certain  level  of  knowledge  for  unfamiliar  systems.  However/ 
your  judgment/experience  is  valued  more  since  the  data  was 
taken  from  open  sources  and  may  not  be  exact. 


1. 

F-4E 

F-16 

19. 

Mirage 

3C 

MiG-23 

2. 

MiG- 21 

F-5E 

20. 

F-5E 

MiG- 2 3 

3. 

MiG- 19 

F-14A 

21. 

MiG-19 

F-5E 

4. 

F-15A 

F-4E 

22. 

MiG- 19 

MiG- 2 3 

5. 

MiG- 21 

F-4E 

23. 

F-15A 

F-16 

6. 

MiG-23 

F-15A 

24. 

F-16 

MiG- 2 3 

7. 

F-5E 

F-16 

25. 

F-5E 

F-14A 

a. 

Mirage  3C 

F-5E 

26. 

F-4E 

MiG-19 

9. 

F-15A 

MiG- 19 

27  . 

F-14A 

MiG-23 

10. 

F-15A 

F-5E 

28. 

F-15A 

MiG- 21 

11. 

F-16 

Mirage 

3C 

29. 

F-4E 

Mirage  3C 

12. 

F-14A 

Mirage 

3C 

30. 

F-4E 

F-14A 

13. 

MiG- 19 

F-16 

31. 

F-4E 

F-5E 

14. 

MiG-19 

Mirage 

3C 

32. 

F-15A 

F-14A 

15. 

F-16 

MiG-21 

33. 

Mirage 

3C 

MiG-21 

16. 

MiG- 21 

MiG- 2 3 

34  . 

MiG-19 

MiG-21 

17. 

F-15A 

Mirage 

3C 

35. 

F-4E 

MiG- 2 3 

18. 

F-14A 

F-16 

36. 

F-14A 

MiG-21 
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APPENDIX  III 


CALCULATIONS  AND  EQUATIONS  FOR  THE  SUCCESSIVE  INTERVAL  METHOD 

I.  Solution  to  the  law  of  categorical  judgment  (equation  13) 
1.  Given  the  equation 


B 

=  _2_ 


S, 


If  the  estimates  of  z.  are  added  over  instances 
ig 

(column  sums) ,  equation  1  becomes 


Z  z.  =  B^(  Z 


i=l 


i=l 


-)  -  (  Z 
i=l 


-)  @ 


3.  Think  of  equation  as  a  linear  transformation  of 

an  interval  scaled  variable.  The  first  term  on  the 
^  1 

right-hand  side,  Z  - - - ,  establishes  the 

i=l  +  c 

unit  of  the  scale,  while  the  second  term, 
n  S . 

Z  - ^ -  ,  establishes  the  origin.  Since  the 

H-  o 

unit  and  origin  can  be  arbitrarily  set,  let  the 
n  S. 

origin,  Z  -  =0,  and  let  the  unit, 


n 

Z 

i=l 


2 


+  c 


n. 
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4.  Using  these  relationships  reduces  equation  to 


I  z .  =  n  B  ,  or 


n  z . 

Z  =  B 

i=l  ^  9 


@ 


5.  Equation  (5)  says  that  an  estimate  of  the  category 
upper  bound/  B^,  is  obtained  from  the  column  average 
of  the  z-score  array. 

6.  Since  estimates  for  z.  and  B  now  exist,  the  only 

ig  g 

thing  required  to  solve  equation  1  is  an  estimate  of 
1 


the  variance. 


To  do  this,  it  is  first 


a .  +  c 

1  _ 

necessary  to  find  the  row  average,  z^.  If  there  are 
m+1  categories. 


Zi  =  E 
g=l 


m  B 
g=l  “ 


m  S . 
E 

<3-1  “ 


[Bg  -  S.  1 


1,2, 


@ 


where  B^  is  the  average  of  the  column  averages. 


7.  From  equations  and 


B  -  B 

 2 


ig 
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8.  Squaring  both  sides,  adding  over  categories,  and 


rearranging  terms 


—  2 

2  (B^  -  B)^ 

'^ig  - 


i  =  1,  .  . 

g  =  1,  .. 


© 


9.  Since  S,  =  Bg  -  z.g  o.  +  c 


s. 

1 


ig 


m 

I 

m 

I 

g=l 


(Bg  -  B)2 


n 

m 


10.  Or,  if  A  is  the  sum  of  the  squares  of  the  columns, 
and  is  the  sum  of  the  squares  of  the  rows,  the 
scale  value  of  each  instance  (row)  is. 
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II.  The  solutions  identified  in  Part  I  are  now  used  to 
obtain  the  results  reported  in  Chapter  IV. 

1.  Step  1;  Arrange  raw  frequency  data  in  a  table  where 
rows  are  instances  and  columns  are  categories,  with 
column  1  being  least  favorable,  etc. 


MiG-19 

15 

15 

4 

MiG- 21 

3 

15 

16 

MiG- 2 3 

9 

11 

14 

F-4E 

9 

16 

9 

F-5E 

6 

18 

10 

Mirage-3C 

3 

16 

15 

2.  Step  2:  Compute  the  relative  cximulative  frequencies 
for  each  row  and  record  them  in  a  new  table.  Exclude 
the  last  column  since  it  will  be  a  unit  column  vector. 


MiG- 19 

.441 

.882 

MiG-21 

.088 

.529 

MiG-23 

.265 

.588 

F-4E 

.265 

.735 

F-5E 

.176 

.706 

Mirage- 3C 

.088 

.559 

3.  Step  3;  Treating  these  values  as  leftward  areas 

under  a  N(0,1)  curve,  find  the  standard  normal  deviates 
for  these  areas  and  record  them  in  another  table. 

This  array  will  have  one  less  column  than  the 
original  array  in  step  1. 
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MiG-19 

-.148 

1.186 

MiG-21 

-1.354 

.073 

MiG-23 

-.625 

.223 

F-4E 

-.625 

.628 

F-5E 

-.931 

.542 

Mirage-3C 

-1.354 

.148 

Step  4:  Compute  a  row 

average  for 

each  row 

/  a 

column  average 

for  each  column. 

a  grand  average. 

B,  and  the  sum 

of  the 

squares  of 

the  columns,  A. 

Row  Sum 

z . 

1 

MiG- 19 

-  .148 

1.18  6 

= 

1.038 

.519 

MiG- 21 

-1.354 

.073 

= 

-1.281 

-.641 

MiG-23 

-  .625 

.223 

= 

-  .402 

-.201 

F-4E 

-  .625 

.628 

= 

.003 

.002 

F-5E 

-  .931 

.542 

= 

-  .389 

-.184 

Mirage- 3C 

-1.354 

.148 

= 

1.206 

-.603 

Column  Sum  = 

-5.037 

+  2.8 

_ 

>  Grand 

Average 

B  =  -. 

186 

-  .839 
t 

.467 

t 

1 

®1 

1 

^2 

A  =  (-.839  +  . 

186) ^  + 

(.467  +  .186) 

^  =  .852 
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5.  Step  5;  For  each  row  compute  . 


=  Mig-19  =  (-.148  -  .519)^  +  (1.186  -  .519)^  =  .890 
=  MiG-21  =  (-1.354  +  .641) ^  +  ( .073  +  .641) ^  =  1.018 
C3  =  MiG-23  =  (-.625  +  .201)^+  (.223  +  .201)^  =  .360 

=  F-4E  =  (-.625  -  .002)^  +  (.628  -  .002)^  =  .785 
C3  =  F-5E  =  (-.931  +  .184) ^  +  ( .542  +  .184) ^  =  1.085 
Cg  =  Mirage-3C  =  (-1.354  +  .603) ^  +  (.148  +  .603) ^  =  1.128 


6 . 


Step  6;  For  each  row  compute 
estimate  of  the  variance. 


This  gives  an 


MiG-19  =  ^11^  =  .978 

=  i^=  -515 

=  t|§  =  1-S4 

F-4E  =  -=^11  =  1.042 

=  T7§|5=  -886 


Mirage-3C 


.852 

1.128 


.869 


7.  Step  7;  Compute  the  scale  values,  S^,  according  to 
the  equation,  S.  =  B  - 
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MiG-19 

= 

-.186 

-  ^ 

.519(.978)  = 

^.694 

MiG- 21 

= 

-.186 

+  , 

.641(.915)  = 

.401 

MiG-23 

= 

-.186 

+  , 

.201(1.54)  = 

.124 

F-4E 

= 

-.186 

.002(1.042)  = 

-.188 

F-5E 

= 

-.186 

+  , 

.184(.886)  = 

-.023 

Mirage- 

3C 

=  -. 

186 

+  .603(.869) 

=  .338 

8.  Step  8 ;  The  scale  can  be  transformed  by  a  linear 

transformation  to  best  suit  the  needs  of  the  analyst. 
In  this  instance  all  values  are  transformed  so  that 
the  MiG-19  has  a  score  of  1.0  by  adding  1.694  to 
every  score. 

MiG-21  =  2.095 

Mirage  3-C  =  2.032 

MiG-23  =  1.818 

F-5E  =  1.671 

F-4E  =  1.506 

MiG-19  =  1.000 
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FIGHTER  CAPABILITY  MEASUREMENT 


The  p\irpose  of  this  questionnaire  is  to  gather  data  on 
the  capability  of  fighter  aircraft.  You  are  asked  to  cate¬ 
gorize  each  of  the  nine  aircraft  listed  below  into  one  of 
five  categories  -■  excellent,  above  average,  average,  below 
average,  poor  -•  on  the  basis  of  their  serial  combat  capa¬ 
bility.  Assume  a  combat  environment  of  5-2 OK  ft  with  speeds 
ranging  between  M.5  -  1.5.  Also  assume  that  no  long  range 
missiles  are  available  and  that  aircraft  are  piloted  by  men 
of  approximately  the  same  ability. 

If  in  doubt  about  the  performance  of  a  certain  aircraft, 
use  the  data  provided.  However,  your  judgment/experience 
is  valued  more  since  the  data  was  taken  from  open  sources 
and  may  not  be  exact. 

EXCELIENT  ABOVE  AVERAGE  AVERAGE  BELCW  AVERACE  POOR 

MiG-19 

MiG-21  PFM 

MiG-23 

F-4E 

F-5E 

F-14 

F-15 

F-16 

Mirage- 3C 
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APPENDIX  IV 


Multi-Attribute  Utility  Calculations 


I .  Level  I  Calculations 


A.  Determination  of  platform  utility  UpCx^). 

Weights  assigned  by  judge  for  platform  criteria; 


k 


k. 


Ws 


‘r 


.35 


.10 


n 


Additive  function  is  assumed,  u  (x  )  =  Z  k.  u. (x.) , 
n  P  P  i=l  111 

and  Z  k.  =  1.  Therefore,  entering  the  utility 
i=l  ^ 

curves  with  the  weapons  performance  characteristics 
for  each  aircraft: 


values  from  utility  curves 


F-16  =  (1.0) (.15)  +  (.9) (.4)  +  (.35) (1.0)  +  (.1) (1.0)  =  0.96 
F-15A=  (.88)  (.15)  +  (.  8)  (.4)  +  (.  35)  (.9)  +  (.95)  (.1)  =  .86 
F-14A=  (.95)  (.15)  +  (.5)  (.4)  +  (.35)  (1.0)  +  (.1)  (.95)  = 

F-4E  =  (.  84)  (.15)  +  (.  6)  (.4)  +  (.35)  (0)  +  (.1)  (1.0)  =  .47 
F-5E  =  (.63)  (.15)  +  (.25)  (.4)  +  (.35)  (.  6)  +  (.1)  (.8)  =  .48 
Mir.3C  =  (.95)  (.15)  +  (.25)  (.4)  +  (.35)  (1.0)  +  (.1)  (1.0)  =  .69 
MiG-19  =  (.5)  (.15)  +  (.55)  (.4)  +  (.35)  (.88)  +  (.1)  (.8)  =  .68 
MiG-21  =  (.62)  (.15)  +  (.5)  (.4)  +  (.  35)  (.8)  +  (.1)  (.5)  =  .  62 
MiG-23  +  (.75) (.15)  +  (.5) (.4)  +  (.35) (.5)  +  (.1) (.9)  =  .58 
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B.  Determination  of  weapon  utility,  u^(x^) . 

Weights  assigned  by  judge  for  weapons  criteria: 


Kr-  =  -15 

k 

=  .15 

mr 

ms 

o  =  *30 

k 

=  .40 

a .  o . 

g 

Israeli/Egyptian 

calculation 

assume 

II 

ki  u. (X.) 

= 

C.  Miscellaneous  Utility,  u  (x  ) . 

mm 

Weights  assigned  by  judge: 

k  =2 

country 

k  4.  =  .8 

pilot 

Additive  function  holds,  therefore, 

^m^^m^  ~  ^country  country  +  pilot 

Israel  u^^  =  .  2  ( .  35)  +  .  8  ( .  75)  =  .67 

Egypt  u  =  .2(.2)  +  .8(.5)  =  .44 

m 


II .  Level  II  Calculation  for  over-all  aerial  combat  utility 

A.  Multiplicative  relationship  was  determined.  Therefore: 


over-all  =  platform  x  weapons  x  miscellaneous 
aerial  utility  utility  utility 

corbat 
utility 
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B.  Weights  assigned  by  judge: 


.2 


=  .1 


C.  From  Level  I  analysis: 


u^(x  )  for  F-4E  =  .47  u^(x  )  for  MiG-21  =  .62 

P  P  P  P 

Vx  )  =  .5  (assumed)  u  (x  )  for  Israel  =  .67 

w  mm 


Up(Xp)  =  u^(x„)  =  u^(xj  =  1. 


1  +  k  =  (1  +  .2k) (1  +  .Ik) (1  +  .6k) 

.012  k^  +  .2  k^  -  .1  k  =  0 
k  =  -.42  and  -16.25 
always  use  the  root  between  -1  and 
k  =  -.42 


E.  Substituting  the  value  of  k  in  the  utility  expression 
(II-A)  along  with  the  values  from  II-B  and  II-C: 


Israeli  F-4E: 

1- .42[U^(X^)]  =  [1  -  .084(.47)]  [1  -  .042  (.5)  ]  [1  -  .25(.67)  ] 

W  =  ^ 

Egyptian  MiG-21: 

1- .42[U^(}^)]  =  [1  -  .084(.62)]  [1  -  .042(.5)  ]  [1- .25(.44)  ] 

W  =  ^ 
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